Issues

Clinical Trials - What the Drug Companies Don't Report

The placebo effect and drop-out rates tell an interesting story.

The placebo effect is real. A 2002 University of Toronto study of brain scans revealed that Prozac and a placebo worked on similar areas of the brain (Prozac, though, worked on more areas of the brain).

With antidepressant drug trials, the placebo effect is high enough to cause a full half of these studies to end in failure, which has set off fierce debate over whether an antidepressant is little more than a placebo with side effects. The focal point of the controversy are two studies by Irving Kirsch PhD of the University of Connecticut:

A meta-analysis of nineteen nineteen double-blind antidepressant trials published in the American Psychological Association's online publication, Prevention and Treatment (Guy Sapirstein PhD of Westwood Lodge Hospital, Needham, MA, co-author) in 1998 caused an uproar in professional circles when it was revealed that the placebo effect accounted for a mind-boggling 75 percent of an antidepressant's result - any antidepressant, you name it.

Four years later, the July 2002 Prevention and Treatment published another study by Dr Kirsch that analyzed the FDA database of 47 placebo-controlled short-term clinical trials involving the six most widely prescribed antidepressants approved between 1987 and 1999. These included "file drawer" studies, ie trials that failed but were usually never published.

What Dr Kirsch and his colleagues found was that 80 percent of the medication response in the combined drug groups was duplicated in the placebo groups, and that the mean difference between the drug and placebo was a "clinically insignificant" two points on both the 17-item and 21-item Hamilton Depression Scale, regardless of the size of the drug dose. The placebo factor ranged from a high of 89 percent for the Prozac response, according to the study, and a low of 69 percent for the Paxil response. In four trials, the placebo equaled or achieved marginally better results than the drug. In the nine expert commentaries published with the study, none of the commentators disputed the study's main findings.

Breaking Down the Study's and Commentaries' Main Arguments

Argument: Drug trials are underwritten by the pharmaceuticals under the most favorable conditions and administered by clinicians with a stake in the outcome, suggesting a strong bias in favor of the drug.

Counter-argument: Moderately-ill outpatients who have the wits to fill out the paperwork and motivation to get themselves to a clinic are the ones who wind up in trials. These patients are much more likely than severely-depressed patients to respond to placebo treatment. Suicidal patients are excluded from trials. Underpowered studies guarantee a 50 percent failure rate. In seeking to enroll patients for a trial, clinicians are apt to inflate baseline Hamilton scores (which if true opens up its own separate can of worms).

Argument: Three study practices that favor the drug - 1) "Inert" rather than "active" placebos, resulting in patients figuring out which group they're in; 2) Early "washout" phases in trials that eliminate both high placebo responders and antidepressant nonresponders; 3) Allowing sedatives to be used concurrently with the test drug.

Counter-argument: One study practice that favors the placebo - A statistical fiction called "Last Observation Carried Forward" (LOCF) that keeps dropouts in the study. In addition, placebo response rates have increased over two decades, making it difficult to show treatment effect, particularly when the placebo response rate is 40 percent or higher.

The "counter" forces also note that high placebo response should not be equated with antidepressant ineffectiveness. As well as a pill, the placebo includes an expert healer and treatment elements that mobilize hope, they point out.

Finally, in measuring "average" effects, the "counter" forces submit that the study ignores the reality of a significant subgroup of patients who would have derived considerable benefit from their antidepressants, not to mention patients who never should have been treated with these drugs. The problem is we have no clue who they are.

How Significant Is Insignificant?

In his expert commentary, Michael Thase MD of the University of Pittsburgh focused on the percentage figures in the study while Dr Kirsch emphasized the two-point Hamilton Scale differential. Allowing that an antidepressant may have only a 10 percent remission advantage over a placebo, Dr Thase argued this still means one in ten patients would remit, a much more meaningful effect than cholesterol reduction would have on heart attack prevention or smoking cessation on lung cancer prevention. Dr Kirsch, however, cited the hypothetical of two study patients with Hamilton scores of 20. The first patient's score is reduced by 10, the second by nine. Technically, the first patient is a responder while the second patient is a nonresponder, but the one point difference in real world terms is insignificant, Dr Kirsch contends.

Particularly Worrying

In his commentary supporting Dr Kirsch's findings, David Antonuccio PhD of the University of Nevada and colleagues cite a 1999 study by Dr Thase that found antidepressants were significantly more effective than placebos in about two-thirds of published trials. The FDA database, however, yields a truer figure of about half, which poses "a serious problem, because clinical practice guidelines based on the published literature fail to take this distortion of the literature into account."

The Long Run

Walter Brown MD of Brown University raised the argument that a placebo is better at alleviating depression than it is at preventing relapse, and that over the long term (not the eight weeks or less in the trials in question) among patients who have improved on antidepressants, it is not uncommon to find a relapse rate of 10 percent over one year for those continuing on their antidepressant vs 50 percent for those reassigned to a placebo. Dr Antonuccio, on the other hand, cites studies that indicate 33 percent of remitted patients return to depressive symptoms.

Ignoring the Obvious

Three of the "pro-drug" commentators briefly alluded to the fact that some patients may be more responsive to certain medications than others, but none of them drove home what many of us know from personal experience - that if our initial antidepressant doesn't work, another one - even one chemically similar - might. There are a number of open-label studies published in different issues of The Journal of Clinical Psychiatry, some by one of the commentators, Dr Thase, that indicate the value of switching to drug B when drug A fails, namely:

* A 1995 study found that of 113 patients who discontinued Prozac due to side effects, 71.8 percent were “much” or “very much” improved after eight weeks on Zoloft, with 9.8 dropping out due to adverse effects.
* A 1996 study found 28 of 55 patients who failed on their first SSRI had a “marked or complete antidepressant response” on their second SSRI.
* A 1997 study of 106 patients who either did not respond to Zoloft or were intolerant of the drug found 63 percent responded to Prozac.
* A 2001 study of 58 nonresponders to Prozac found 63 percent responded to Celexa after 12 weeks, with 18 percent dropping out.

A double-blind study (again with Dr Thase as one of the authors) published in the March 2002 Archives of General Psychiatry illustrates the benefit of switching across classes of antidepressants. Of 117 chronically depressed patients who failed to respond to Zoloft after 12 weeks and 51 patients with similar bad luck on imipramine, more than half from each group benefited from switching to the other.

These finding have since been corroborated in part by the NIMH-underwritten STAR*D real world depression trials published in 2006.

Even the champions of drug therapy acknowledge that our first antidepressant basically amounts to scratch lotto in pill form. But our next attempt or the attempt after, like reverse Russian roulette, may yield the bullet, maybe not the magic bullet we are all so desperately seeking, but something reasonably serviceable that gives us a fighting chance of recovery.

On the Other Hand

The above scenario assumes educated patients in the hands of expert clinicians. The reality tends to be scared and confused patients at the mercy of primary care physicians who think Prozac is some kind of Aspirin.

An eye-popping finding of equal import, which the study's authors Irving Kirsch PhD and Thomas Moore PhD are making in public presentations, is that only 63 percent of those taking antidepressants in the pooled drug studies completed the four to eight-week trials. The combined placebo group recorded a similar figure of 60 percent. Only four of the studies achieved the 70 percent completion rate sought by the FDA.

In other words, nearly 40 percent of those taking antidepressants failed to make it through the four to eight weeks of a normal drug trial. This figure dwarfs the numbers the drug companies in the study put on their product information, namely:

Prozac 12 percent; Paxil 20 percent; Zoloft 14 percent; Effexor XR 11 percent; Serzone 16 percent; Celexa 15 percent.

An important qualifier is the drop-outs in the separate drug studies were due to adverse events only, but the results fare miserably against other classes of drugs using the same measure. For example, only 1.4 percent of patients discontinued the cholesterol drug Zocor due to adverse events, seven percent quit the arthritis drug Celebrex (which had much longer trials), and just 1.5 percent said no to the heart drug Norvasc.

As to what constitutes an unacceptable drop-out rate in other branches of medicine, AstraZeneca reported ending a 1997 study of its breast cancer drug Nolvadex (tamoxifen) due to 26 percent of patients quitting, but this was after one year. A 26 percent drop-out rate in a year-long psychiatric drug study, by contrast, would have its manufacturer trumpeting the medication's safety and tolerability. Oddly enough, a small pilot study recently tested tamoxifen for mania. It will be interesting to compare drop-out rates if larger studies are done.

[Two weeks after publishing this, Lundbeck AS, which manufactures the antidepressant Lexapro in Europe, announced the completion of a "successful" 52-week study in which - yes - 26 percent of the patients dropped out.]

Meanwhile, it is instructive to take a closer look at the Paxil figures. Paxil represents a virtual Swiss Army knife of antidepressants, with a use for any number of psychiatric situations, FDA approved. Whereas 20 percent of Paxil's depression patients dropped out of trials due to adverse events, according to the manufacturer's data, only nine percent of those with panic disorder quit, followed by 11 percent for generalized anxiety disorder, 12 percent each for OCD and PTSD, and 16 percent for social anxiety disorder.

In theory, these numbers should have been about equal across all groups, but depression sufferers clearly found much more not to like besides just side effects, such as maybe their meds not working. Doses are typically started at the low end of the scale, with the clinical benefit not beginning to show until the second week and a meaningful response not occurring until the fourth to sixth week or longer, if at all. Perversely, the side effects make themselves felt at once. Not exactly an inducement to stay on one's meds, not even the ones that come as orange-flavored sol-tabs.

So what is the most meaningful figure in an antidepressant trial? Apparently not the response rate, not the remission rate, not the Hamilton Depression scores. It's the drop-out rate, way too high whether going by industry figures or the FDA database. Clearly we are sending an unequivocally strong message that our medications leave much to be desired. Are any drug companies listening?

Updated Feb 12, 2008