Men fear 'chick-lit' and read far fewer books by female authors, says Women's Prize judge