White House keeping access to Oval Office limited - chief of staff