An alleged child abuser died before his trial. His accusers are now being blamed