Publication Bias: Not Dead Yet

Suppose you do two clinical trials of a drug, and only one of them shows it to work. It would be entirely misleading to only tell people about that one, and sweep the negative result under the carpet - but it happens.

That's publication bias. A simple but powerful remedy is to require everyone to publically announce their trials before the data comes in. The USA has led the way in this, with the public clinicaltrials.gov database, and for several years it's been a legal requirement that all clinical trials conducted in the USA must be pre-registered there, and that the results have to be uploaded when they arrive.

A new study by Bourgeois et al used this database to assess the scale of non-publication: Outcome Reporting Among Drug Trials Registered in ClinicalTrials.gov. Out of over 500 clinicaltrials.gov trials they looked at, 66% of them ended up getting published eventually. (The trials all ended before 2006, so they've had 5 years to get published and if they haven't by now, it's unlikely they ever will.) Is that a lot? Well, it's better than I'd expected, but it's still 33% too low.

The odds of getting published varied depending upon the type of drug, though. Trials of proton-pump inhibitors and cholesterol lowering drugs had the best chances. Antidepressants were a bit less publishable; antipsychotics were markedly worse; and only just over half of the vasodilator trials did.

This is interesting data, and it should remind us that publication bias, although often discussed nowadays as a problem with trials of antidepressants, is by no means limited to those drugs and in fact antidepressant trials (at least the ones starting after 2000 and completed by 2006) are fairly middle-of-the-road in terms of % publication rates.

Publications resulting from drug company-funded trials were also more likely to be positive (85%) than were trials bankrolled by the government (50%) or non-profit organizations without Pharma "contributions" (61%). This doesn't prove that drug companies are biasing publication - maybe they just really do get more positive results - but, well, it's not exactly reassuring.

Why is non-publication still a problem, given that people are required by law to release their trial protocols and results on clinicaltrials.gov? The problem is that clinicaltrials.gov doesn't appear on PubMed, and medical science works on the rule of "PubMed or it didn't happen". Someone searching for papers about "drug X for disease Y" - which I suspect accounts for the vast majority of clinical paper downloads - will still only get told about the trials that the authors chose to publish.

Is there an answer? We could in theory force people to write their results up and submit them to a journal, and we force journals to publish them, but that would be unworkable and incredibly unpopular. But why not simply publish the results from clinicaltrials.gov?

Whenever someone uploads their results to the database (as they legally must), clinicaltrials.gov could automatically use them to generate a mini-paper and publish it online. There would be a few tricky issues to sort out - you'd have to be careful that it didn't lead to the same results getting published in multiple places, for one - but so long as these reports were indexed on PubMed, it would solve the fundamental problem.

Bourgeois FT, Murthy S, & Mandl KD (2010). Outcome Reporting Among Drug Trials Registered in ClinicalTrials.gov. Annals of internal medicine, 153 (3), 158-66 PMID: 20679560