Yet West added the 2019 study, published in Aggressive Behavior and led by psychologist Qian Zhang of Southwest University of Chongqing, to his meta-analysis after a reviewer asked him to cast a wider net. West didn’t feel his vague misgivings could justify excluding it from the study pool. But after Aggressive Behavior published West’s meta-analysis last year, he was startled to find that the journal was investigating Zhang’s paper while his own was under review.
It is just one of many papers of Zhang’s that have recently been called into question, casting a shadow on research into the controversial question of whether violent entertainment fosters violent behavior. Zhang denies any wrongdoing, but two papers have been retracted. Others live on in journals and meta-analyses—a “major problem” for a field with conflicting results and entrenched camps, says Amy Orben, a cognitive scientist at the University of Cambridge who studies media and behavior. And not just for the ivory tower, she says: The research shapes media warning labels and decisions by parents and health professionals.
The investigations were triggered by Illinois State University psychologist Joe Hilgard, who published a blog post last month cataloging his concerns about Zhang’s work. Hilgard was initially impressed when he came across a 2018 paper of Zhang’s in Youth & Society, another study with 3000 subjects. “I was like, holy smokes!” he says. The study found some teenagers were more aggressive after playing violent video games. Given the huge sample size, it had the potential to be a “powerful chunk of evidence,” Hilgard says.
But he found the paper’s statistics mathematically impossible. Zhang and his co-authors reported high levels of statistical significance for their finding, but the reported differences in the effects of violent games versus nonviolent games were too small for that high statistical significance to be possible. Hilgard alerted Zhang and the journal, and Zhang submitted a correction. Hilgard says that made the statistics seem more plausible, but they were still incorrect.
Hilgard says he found problems in other papers of Zhang’s, such as nearly identical results reported in three different papers. He emailed Zhang and asked to see his data, but he says Zhang refused. Hilgard then contacted Dorothy Espelage, a psychologist at the University of North Carolina, Chapel Hill, and co-author with Zhang on multiple papers. She told Hilgard that Zhang had refused to send her the data, too. It was only after Hilgard asked Southwest University to investigate that Zhang sent Hilgard data for a Youth & Society paper on movie violence.
But the data were odd, Hilgard says, and missing features normally found in similar experiments. He sent his findings to Zhang’s university, which said it found problems, but not fraud, with Zhang’s data, and that Zhang was “deficient in statistical knowledge and research methods.” Dissatisfied with the response, Hilgard sent his observations to all the journals involved.
Modern science has a lot more bad research than it should, because the rewards for research are so high, and not especially dependent on its quality.
