Trial registration can cure dangerous publication bias
By Sarah Lovinger, ACP member
Physicians expect that the knowledge they bring to clinical practice is based on the best evidence available. But what happens to patient care when important scientific research does not get published? Doctors may be deprived of critical information, and patients may suffer.
Kay Dickinson, PhD
In the absence of a global system for registering clinical trials, journal editors may be selective about what trials, or parts of individual trials, they publish. Information on negative outcomes can be hidden from doctors in the absence of an international trial registry to ensure complete transparency in scientific publishing. This kind of selective reporting, also known as publication bias, can lead to the underreporting of important adverse effects, such as suicidality in adolescents taking SSRIs.
Kay Dickersin, PhD, professor in the Department of Epidemiology and director of the Center for Clinical Trials at Johns Hopkins Bloomberg School of Public Health, is widely considered an expert on publication bias. In addition to her individual research, she is also director of the U.S. Cochrane Center and co-chairs the Scientific Advisory Group of the International Clinical Trials Platform of the World Health Organization. She recently spoke to ACP Observer.
Q: How do we define publication bias?
A: It is the tendency on the part of authors, editors, and reviewers to selectively submit and accept manuscripts for publication on the basis of the strength and direction of the study findings.
Q: How do we know that publication bias occurs?
A: There have been many studies showing that not all studies are published. About 10 studies following research approved by an ethics committee or funded by government have all shown the same thing: positive findings, frequently defined as statistical significance, are related to publication. So if you are just looking at the published literature, you should be aware that some studies have only reached the abstract stage, and some have not even reached the abstract stage.
Q: How prevalent is publication bias?
A: One estimate is that positive results are three times more likely to be published than null or negative findings. We know that only about one-half of all studies that reach the abstract stage get published in full. People have looked at whether funding is related to publication bias, and it is. Publication bias is stronger when studies are funded by industry than when they are funded by other sources.
Q: What are the effects on scientific reporting and thus on clinical practice?
A: At the extreme, this might mean that we think a drug works when it does not. For example, a trial done in 1980 showed an increased mortality risk in people taking a class I antiarrythmic drug. The authors disregarded this finding initially, and the findings from this trial were only added to our knowledge base 13 years later when the results were reported. This was after many people who took this drug died.
Sometimes publication bias occurs because the authors don't report results. The association between SSRIs and suicidality in adolescents was not known [initially] because these findings were not published. [It appears that the link between] Cox-2 inhibitors and cardiovascular events was due to deliberate withholding of information. In these cases, there was what we call selective outcome reporting.
Q: How does publication bias affect meta-analyses and how can researchers take the possibility of publication bias into consideration when conducting a meta-analysis?
A: When conducting a systemic review, authors should perform extensive electronic searching and even some hand searching of key periodicals (for example, conference abstracts). They would also look in bibliographies of published articles, contact experts in the field, and work with industry to do a thorough search to find published and unpublished information. Researchers can also do a funnel plot, and if they see an absence of study findings in part of the funnel, they might conclude that there is publication bias. There are other methods to detect publication bias, but none of them are great.
Q: What changes could be made to improve systematic reviews?
A: Comprehensive trials registration is desperately needed. A single portal to link all existing trials registers is a good start. This would not provide trial results, but at least we would know a trial has been done.
If readers are interested in finding out how well an intervention works, they should look for a systematic review. Not all systematic reviews are created equal however! Readers should assess the quality of the systematic review. For example, if the review simply searched the English language literature it could have missed important studies, some with negative results. With a trials registry one could tell if any studies were missing from the review. Readers should actively support trial registration. If a trial was not registered, [it's important to know] why not.
Q: What progress has been made in promoting clinical trials registration?
A: In the U.S., we now have http://clinicaltrials.gov, in which close to 100% of NIH funded trials have been registered since 2002. Perhaps as many as 50% of industry-sponsored randomized trials for serious and life-threatening diseases are registered there. The requirement of international medical journals to register trials was very effective in getting more industry trials registered. Industry does not want to miss out on the opportunity to publish positive trial findings in good journals. Unfortunately, the decision to require trial registration prior to publication is made at the individual journal level Small and specialty journals may feel that they can't take a risk of requiring registration—they may fear losing submissions.
Q: What is the overall outlook for making publication bias less prevalent?
A: In the U.S. I would say the outlook two summers ago for trial registration was excellent. But now, Congress is not doing anything about it and they haven't in a while. Consumer groups that were very involved appear to have dropped the ball. It's going to take another tragedy for people to get interested—for example the SSRI uproar involved death and children.
Q: How can interested people promote activities countering publication bias?
A: The Society for Clinical Trials has endorsed [a clinical trials registry]. If more professional societies were involved, I think we might be able to get somewhere. Industry is very strong and it has not supported trials registration. It's very important for them to keep [some] information quiet and they are clear about that. PhRMA has been active in asking for delayed disclosure of key information—outcomes examined, intervention tested, sample size, even the scientific title of the study. If we agree to industry's demands, we are choosing commercial interests over science and ethics. I don't think that scientists or consumers as a group have done their part to support their interests so far.
Sarah Lovinger, ACP Member, is a free-lance medical reporter based in Evanston, Ill., and a general internist who practices part-time in Chicago.
The information included herein should never be used as a substitute for clinical judgment and does not represent an official position of ACP.
Internist Archives Quick Links
Have questions about the new ABIM MOC Program?
One Click to Confidence - Free to members
ACP Smart Medicine is a new, online clinical decision support tool specifically for internal medicine. Get rapid point-of-care access to evidence-based clinical recommendations and guidelines. Plus, users can easily earn CME credit. Learn more