PM says people understand why taxes cannot be cut as they're not 'idiots'