While so-called 'negative studies' may not make the most compelling headlines in medical journals, a lack of balance between studies that do and don't trigger a change in the way physicians practice can warp the way readers view the research that does make the cut, according to a report in the Nov. 22 Archives of Internal Medicine.
"No-difference studies affect practice just as much as positive ones, but they aren't as sexy," said Dr. Seth Leopold of the University of Washington in Seattle, who led the research. "Something splashy, something new, is more exciting to everybody."
With the vast majority of journal articles focused on the next best thing, the authors argue, it can make the new drugs and treatments seem better than they really are, while giving the proven standbys none of the respect they deserve.
To study the level of 'positive-outcome bias' among journal editors, researchers presented peer reviewers at two orthopedic journals with one of two fictitious studies that were identical in every way except one: One study showed that one treatment was superior to another, while the other indicated no difference between treatments. Reviewers not only overwhelmingly picked the positive study (97.3 percent recommended publishing the positive study vs. 80 percent for the negative study), but they also more harshly criticized the quality of the no-change study. In reality, the same five errors were intentionally included in both manuscripts, with each study's methodology being exactly the same. Leopold said the findings suggest reviewers were looking for reasons to endorse positive findings and reject negative ones.
While acknowledging that researchers may be more likely to submit positive studies for publication in the first place, Leopold and colleagues suggested that journals should encourage authors to submit high-quality studies with negative results and give them higher priority for publication, perhaps in nontraditional forms such as online appendices.
To learn more:
- read the article in MedPage Today
- see the Reuters story from Fox News
- find the study in the Archives of Internal Medicine