Frank James was quiet and gruff, his neighbors in Milwaukee said.