What You Don't Know About a Drug Can Hurt You

Untold Numbers of Clinical-Trial Results Go Unpublished; Those That Are Made Public Can't Always Be Believed

The Wall Street Journal December 15, 2008
By ROBERT LEE HOTZ

There's a common assumption that when a drug makes it to market, it has run a rigorous gantlet of testing and proper disclosure. Testing, yes. Disclosure -- not necessarily.

Findings from many clinical studies assessing prescription drugs never see light of day. That skews the basic scientific record that every patient, physician and researcher needs to judge whether treatments cause more harm than good. There is no easy way to discover how much knowledge we've been missing, raising the possibility that we may be taking medications that are less effective than we've been led to believe or may have undisclosed side effects.

Forum

Discuss: Do you trust prescription drugs? Do you think pharmaceutical research is over-regulated or under-regulated?

"There is data that you are not seeing," says Deborah Zarin, director of the National Library of Medicine's online clinical trials registry. "There is a huge problem here if an unknown amount of the information is censored and you don't even know it is censored."

All told, about $58 billion in privately funded drug research is currently underway -- twice the basic federal medical research budget -- encompassing an estimated 50,000 clinical trials among 2.3 million patients. Even under the best of circumstances, though, the sheer volume of clinical testing has overwhelmed medicine's ability to independently assess these commercial trials and make all their results public, medical research analysts have concluded.

For a decade or more, researchers, federal regulators, pharmaceutical companies and medical journal editors have sought a balance between commercial secrecy, scientific openness and the public health. Incomplete test results can lull doctors into believing that drugs they prescribe are better than a placebo, and mislead scientists conducting follow-up studies.

Since 2004, prescription drug scandals over deceptive marketing, unreported side effects and hidden payments to medical researchers have highlighted the gap between the number of clinical trials conducted and the number published. That has prompted reforms in how commercial medical data is handled. The Pharmaceutical Research and Manufacturers Association of America, for example, created an online data base that summarizes clinical study results involving hundreds of prescription medicines.

Findings from many clinical studies assessing prescription drugs never see light of day.

By law, comprehensive clinical data must be filed at the U.S. Food and Drug Administration, to make the case for market approval of a new drug and form the basis for safety information that accompanies every prescription. All the critical information that a doctor or patient needs is summarized on the FDA-approved drug label, says Ken Johnson, senior vice president of the pharmaceutical research trade association, "including important drug safety information that may not be included in a published paper." Such information is updated regularly as new findings become available, industry experts say. "It is important for people to know that the FDA sees all of the data," says Jeffrey Francer, the trade association's assistant general counsel.

After years of congressional debate, however, the FDA now has imposed more stringent reporting requirements to ensure the complete scientific record of clinical drug experiments is readily available. Under a new federal law, researchers for the first time will have to post their basic results publicly on the federal online registry maintained by the National Library of Medicine.

Since it was established nine years ago as a voluntary patient guide, the federal registry at www.ClinicalTrials.gov has logged 65,000 studies. Prodded by medical journal editors and settlements in five major lawsuits, researchers have gone from registering 25 new studies a week to about 350 every week. Until this past September, though, researchers only had to report the start of a clinical trial. They had no federal obligation to report the outcome in the registry or in a peer-reviewed journal.

Several recent surveys of the medical literature offer a glimpse of how much data about drugs already on the market never made it into the published record.

Last month, analysts led by health research expert Lisa Bero at the University of California in San Francisco checked 164 clinical trials testing 33 different drugs submitted for FDA approval from 2001 to 2002 and found that one in four had yet to be published. Almost all of the unpublished findings made the drug in question look bad.

Recommended Reading

Under a new federal law, researchers conducting clinical trials must post their findings publicly at an online federal clinical trials registry maintained by the National Library of Medicine. The Pharmaceutical Research and Manufacturers Association of America also maintains summaries of clinical study results in an online data base.

Normally, clinical trials involving safety and effectiveness of new drugs are reported to the U.S. Food and Drug Administration but are not always published in peer-reviewed medical literature, University of California at San Francisco medical analysts reported in "Reporting Bias in Drug Trials Submitted to the Food and Drug Administration: Review of Publication and Presentation," published in the journal PLoS Medicine. Medical informatics experts at UCSF found that more than half of all supporting trials for FDA-approved drugs remained unpublished in "Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature Analysis," in PLoS Medicine.

Writing in The Oncologist, researchers reported that many clinical trials of cancer chemotherapy drugs are not published, in "Practicing on the Tip of an Information Iceberg? Evidence of Underpublication of Registered Clinical Trials in Oncology."

In Drugs: From Discovery to Approval, author Rick Ng offers a step-by-step introduction to all the key processes involved in bringing a drug to the market, including the performance of pre-clinical studies, the conduct of human clinical trials, and regulatory controls.

In The Truth About the Drug Companies: How They Deceive Us and What to Do About It, former New England Journal of Medicine Editor Marcia Angell explains how much of the revenue generated by "Big Pharma" goes into aggressive marketing campaigns.

In Understanding Pharma: The First, Practical Guide on How Pharmaceutical and Biotech Companies Really Work, John Campbell covers pharmaceutical research and discovery, clinical development, marketing, sales, and business development.

In September, medical informatics expert Ida Sim at UCSF and her colleagues reviewed 900 FDA filings involving 90 new drugs and reported that more than half of the clinical trials were still unpublished 5 years after the drugs had been approved.

Only one in five cancer clinical trials ever is disclosed in a medical journal, doctors at the University of Washington reported in the journal Oncology earlier this year.

"What's happening in oncology is happening in all other fields of medicine," says study co-author health economist Scott Ramsey at the Fred Hutchinson Cancer Research Center in Seattle, who checked more than 2,000 chemotherapy trials recorded in the federal registry. "You may not get a full picture of whether a drug is effective or not. With the stakes being what they are in terms of money and human lives, this is a big problem in my view."

Moreover, when clinical data is published in one of 5,200 peer-reviewed biomedical journals, the information on health risks and benefits may be biased by spin and selective reporting, medical analysts said.

For example, researchers who reviewed the FDA's regulatory paperwork for dozens of recently approved drugs found that in some clinical trials submitted for publication, conclusions had been changed, statistics revised, and outcomes altered to make treatments look more effective. Among 43 outcomes reported in the FDA filings that did not favor a drug, 20 were never published. In four out of five instances in which the statistical significance of findings was changed from the FDA filing, the published version was more favorable.

"We saw a lot of changes," Dr. Bero says. "All the changes that occurred made the drug look more favorable."

That reflected a broader pattern of medical research documented by Dr. Bero and other analysts. A study showcasing the promise of a new drug was more likely to be submitted for publication than one reporting negative findings. "It was not that journals are suppressing the information," Dr. Bero says. "People were not submitting the data."

Earlier this fall, researchers at the State University of New York reviewed 1,835 clinical research articles from four leading otolaryngology journals and reported that a third of them failed to mention any side effects at all. In half of those studies that did mention harmful effects, researchers failed to describe them.

"The evidence from clinical trials conducted by the companies that produce the drugs is really the only source of evidence for whether a drug works or not," Dr. Sim says. "The public deserves a fair representation of the evidence for and against expensive new drugs."

No one knows why so much test data was suppressed, overlooked or discarded. In the competitive hustle of academia and commerce, the public interest can just take second place to other considerations. "Journals want to sell journals; companies want to increase profits; and researchers would like to advance their careers," says Dr. An-Wen Chan, a medical policy analyst at the Mayo Clinic in Rochester. "They can conflict. Sometimes, decisions are not made in the best interest of the patients."

As of September, failure to report basic data about clinical test results on the federal public registry in a timely way can open researchers to fines of $10,000 a day and loss of their federal research funding. Scientists now are reporting their findings at a rate expected soon to reach 200 new studies every week. "From our perspective, it is very hard to argue now that there is not transparency," says the pharmaceutical association's Mr. Francer.

Even so, the new registry regulation only covers new tests. Registry reports of serious side effects won't be required until next year. The findings also won't be formally verified.

It will be at least another two years before all the new mandatory reporting requirements are in place and it may be a decade before anyone will know whether so much data disclosure actually improves patient well-being and the quality of care.

"The companies are still fighting every step of the way," Dr. Zarin says. "But they have realized there is a public relations advantage in appearing more transparent."

Robert Lee Hotz shares recommended reading on this topic and responds to reader comments at WSJ.com/OnlineToday. Email him at sciencejournal@wsj.com

---

Newsletter

Did you miss an issue of our online newsletter? You can access past newsletter issues or you can receive future newsletters via email by joining our mailing list.