Antidepressant Drugs versus Placebos

Irving Kirsch is well-known psychology professor who specializes in what is referred to as response expectancy theory.  The basis for this line of research is the perfectly reasonable assumption that what people experience in a given situation is influenced by what expectations they have.  Such considerations are very important in clinical tests of medications where comparisons are made between the responses of patients who are provided either the drug under study or an inert substance, a placebo.  Patients are not told which type of pill they receive.  Often, patients receiving a placebo will claim to have benefited from it.  This is referred to as a “placebo effect.”

Placebo effects are particularly important with psychotropic drugs used in addressing mental illnesses.  There are few if any physical markers for mental illness other than observable behaviors or experiences reported by the patient.  There are no blood tests to analyze, no tumors to measure, and no way to be certain that drug effectiveness is not mostly or entirely a placebo effect.  What are available are clinical tests, usually provided by the pharmaceutical company trying to market the drug under investigation.  The dirty reality of medical science is that these studies are notoriously easy to game in order to produce desired results.  One must be careful in evaluating them.  A researcher named John Ioannidis published an analysis of a collection of some of the best-known, and most widely-accepted clinical studies and concluded that most of them had systematic errors, improper uses of statistical analyses, or researcher bias associated with them, and were thus of suspect validity.  A number of the clinical results could not be reproduced by subsequent studies.  David H. Freedman was motivated by Ioannidis’s work to produce the article Lies, Damned Lies, and Medical Science for The Atlantic in 2010. 

Kirsch became famous and controversial for his studies of the placebo effect with respect to antidepressant drugs.  In 2008, he published an analysis of clinical tests studying the effectiveness of several antidepressant drugs.  This is not a trivial exercise because drug companies rarely publish anything but favorable results.  Nevertheless, they are supposed to provide a complete suite of studies to regulatory agencies like the FDA.  Kirsch gathered as much of the data as he could and concluded that the drugs, while statistically better than a placebo, did not have an effect large enough to be considered clinically significant.  In other words, they didn’t work as advertised. The media outlets picked up on this and proclaimed that “antidepressants don’t work.”

A new analysis was recently published in Lancet (February, 2018): Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis.  The main author on this work was Andrea Cipriani.  This effort analyzed data on more drugs and a much larger sample size of participants.  The published results reached the general public with media claims such as “It’s official, antidepressants work,” and “Study proves antidepressants work.”  The problem with these claims is that the new study actually shows that by one measure, antidepressants are slightly less effective than Kirsch claimed when they were deemed to be useless by the press.  What is going on?

A writer for Discover magazine who goes by the name “Neuroskeptic” provides some insight in About That New Antidepressant Study. 

“The truth is that while the Lancet paper is a nice piece of work, it tells us very little that we didn’t already know, and it has a number of limitations. The media reaction to the paper is frankly bananas, as we’ll see below.”

“Here’s why the new study doesn’t tell us much new. The authors, Andrea Cipriani et al., conducted a meta-analysis of 522 clinical trials looking at 21 antidepressants in adults. They conclude that “all antidepressants were more effective than placebo”, but the benefits compared to placebo were “mostly modest”. Using the Standardized Mean Difference (SMD) measure of effect size, Cipriani et al. found an effect of 0.30, on a scale where 0.2 is considered ‘small’ and 0.5 ‘medium’.”

“The thing is, ‘effective but only modestly’ has been the established view on antidepressants for at least 10 years.”

The author also points out that Kirsch’s negative study actually assigned a slightly greater effectiveness to the drugs, making the public response to the new study rather ridiculous.

“Cipriani et al.’s estimate of the benefit of antidepressants is also very similar to the estimate found in the notorious Kirsch et al.  ‘antidepressants don’t work’ paper! Almost exactly a decade ago, Irving Kirsch et al. found the effect of antidepressants over placebo to be SMD=0.32, a finding which was, inaccurately, greeted by headlines such as ‘Anti-depressants “no better than dummy pills”’.”

Neuroskeptic provides this conclusion.

“Overall, there’s no big surprises here. The new paper confirms what we already knew about antidepressants, and the media confirmed what we knew about the media.”

Neuroskeptic also raised an extremely serious issue that remains unresolved: antidepressant drugs could owe their limited effectiveness entirely to the placebo effect.

“Another caveat is that a meta-analysis is only as good as the data that goes into it, and one concern that hangs over pretty much all antidepressant trials is the issue of unblinding….. According to some people, all of the benefits of antidepressants might just be a placebo effect, driven by people who feel side-effects and then assume that the drug must be working, making them happier. I don’t subscribe to this view but there is very little good evidence either way.”

Given this state of affairs, what does a practicing psychiatrist do?  Daniel J. Carlat provides his view in Unhinged: The Trouble with Psychiatry - A Doctor's Revelations about a Profession in Crisis.

“If you ask any psychiatrist in clinical practice, including me, whether antidepressants work for their patients, you will hear an unambiguous ‘yes.’  We see people getting better all the time.  True, much of this response is undoubtedly due to the placebo effect, but it would be deceptive for me to prescribe a sugar pill to my patients while telling them that it is a real medication.  So I am stuck with prescribing active psychotropic drugs in order to activate the placebo, with the main disadvantage being that such drugs have far more side effects.”

Carlat also provides some interesting insight into the dangers inherent in assuming clinical trials are providing valid information.

“Companies have found by experience that if they want to be sure their drug outperforms a placebo, they have to be very picky about which patients are allowed into the study.  They want patients with ‘pure’ depression, unblemished by messy problems, alcohol use, anxiety problems, or bipolar disorder.  Furthermore, because of reasonable worries about the safety of patients who might be assigned to a sugar pill, these studies exclude patients with suicidal thoughts.  Other common exclusion criteria include an active medical illness, depression that is too mild, or conversely, depression that has lasted too long.”

Carlat wondered whether any of his own patients would have been allowed to participate in one of these studies.

“Mark Zimmerman, a psychiatrist at Brown University had this same thought and decided to test it.  He identified 346 depressed patients who had shown up for treatment at Rhode Island Hospital’s department of psychiatry.  Then he pretended that each one was applying for a spot in a typical antidepressant research study, and applied each of the many exclusion criteria commonly used.  Only twenty-nine out of the 346 patients, or 8.3 percent, would have gained entry into this exclusive club.  The only Ivy League college that matches this is Harvard, which had a 7.1 percent admission rate in 2008.”

This leads to this conclusion from Carlat.

“The bottom line is that antidepressant research studies are not generalizable to real patients, meaning that few of their results, whether positive or negative, are reliable indicators of what would happen to your mood on antidepressants.”

Yep—lies, damned lies, and medical science.