Ban on new rural homes to avoid influx of walkers ‘defies all logic’