The New England Journal of Medicine has just published an interesting and possibly disturbing study showing that the anti-depressants Prozac and Paxil may be less effective than previously thought because studies showing their lack of efficacy were never published or incorporated into overall reviews of the drugs. As the New York Times reports:
In published trials, about 60 percent of people taking the drugs report significant relief from depression, compared with roughly 40 percent of those on placebo pills. But when the less positive, unpublished trials are included, the advantage shrinks: the drugs outperform placebos, but by a modest margin, concludes the new report, which appears Thursday in The New England Journal of Medicine.
Previous research had found a similar bias toward reporting positive results for a variety of medications; and many researchers have questioned the reported effectiveness of antidepressants. But the new analysis, reviewing data from 74 trials involving 12 drugs, is the most thorough to date. And it documents a large difference: while 94 percent of the positive studies found their way into print, just 14 percent of those with disappointing or uncertain results did.
It’s important to note that the authors of this study were not able to investigate the reasons for non-publication:
“We cannot determine whether the bias observed resulted from a failure to submit manuscripts on the part of authors and sponsors, from decisions by journal editors and reviewers not to publish, or both.”
There might be good reasons why some of the trials were not published – poor design for example. And without that kind analysis, which this NEJM study doesn’t provide, the idea that the manufacturers were engaged in some sort of conspiracy to cover up data detrimental to selling their products remains unproven.
Still, the significance in metal analysis of failing to account for studies that find no result was highlighted by the research which claimed the diabetes drug Avandia was associated with an increased risk for heart attack. When studies which didn’t find an association were factored into the meta analysis, this increase in risk disappeared.
Dare we bring up the irony that both the New England Journal of Medicine, which published the Avandia study by cardiologist Steve Nissen, and the New York Times, which gave it significant coverage both on the news pages and on the editorial pages, overlooked this confounding absence, even when it began to be the subject of widespread criticism?