Let’s begin with Leaf’s article.
Leaf describes a study of the drug Avastin for use with patients having a form of brain cancer.
"But to the surprise of many, Dr. Gilbert’s study found no difference in survival between those who were given Avastin and those who were given a placebo."
The smaller studies were performed without comparison groups so that any improvement obtained could be attributed to the action of the drug whether it was or not.
Leaf tells us that many physicians believe, from personal experience, that Avatin was helpful for some of their patients. One could discount the anecdotal data as being unreliable, and, based on the Gilbert results, conclude that the medication was of no value in this case.
Leaf seems to accept the anecdotal claims and draw a different conclusion: the drug works, but only on a small class of patients.
No numbers are presented to support Leaf’s conclusion. Nevertheless, let’s see where this leads. Leaf extrapolates that conclusion and applies it more broadly, suggesting that response to drugs is highly specific to the individual patient and broad studies are not well-designed to recognize these differing medical outcomes.
"Which brings us to perhaps a more fundamental question, one that few people really want to ask: do clinical trials even work? Or are the diseases of individuals so particular that testing experimental medicines in broad groups is doomed to create more frustration than knowledge?"
Leaf then goes even further with this thought and attributes the withdrawal of a number of medications from the market to a lack of understanding of how they work on individuals.
Those readers who only recall the drugs removed from market because they were found to be dangerous and had been approved based on clinical testing that was often shoddy and purposely misleading might be wondering which drugs Leaf was referring to. No information is provided to support the claim.
Taking Leaf’s reasoning to its logical conclusion, a drug that helps 10 people, kills 10 people, and has no effect on 1,000 others is a perfectly good drug. One merely needs to figure out who the 10 people are that it will help and give the drug only to them.
Leaf seems to believe that a major fault in current clinical testing arises from using sample sizes too small to accurately observe effects on the scale of interest.
Let’s now look at Leaf’s claims from the point of view of a drug company. If it could prove that a given drug was very effective, but for only 5% of the population, and that it could detect that 5%, it would have a very strong case for having that drug approved for use. If this were the case, then a pharmaceutical industry struggling to maintain profit growth and to develop new products, would suddenly have an endless supply of new drugs to pursue. If the drug applies only to 5% of the population because of genetic reasons, then there is the potential for many more variations (20?) to address the remaining 95% of the population. Each would, of course, be expensive to develop and come with a hefty price tag, presumably much higher than the price of a single drug that applies to a large population.
Let’s hold that thought for a moment and consider the article by Freedman.
Freedman focuses on the work of the doctor John Ioannidis and his associates. Ioannidis has concluded that the data emerging from clinical trials cannot be assumed to be trustworthy.
That sounds like the opinion of some marginalized crank—but it isn’t.
He is most famous for an article published in the Journal of the American Medical Association.
What did he discover?
Ioannidis has arrived at a rather simple, although startling, explanation for why medical studies are so often wrong or misleading.
Most of the initial data on drug efficacy is produced by the drug companies themselves. Consequently, the bias that Ioannidis sees being introduced has the drug companies as its major source. Ben Goldacre wrote in detail about how this is accomplished in his book Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients. In his chapter describing how to manipulate clinical trials and mislead the medical community he lists 15 techniques that have been used. Misrepresentation of medical results is easy—and it is common.
Bigger and more expensive testing will not help if the trials are improperly designed and analyzed.
The notion that medications can be tailored to the specific physical responses of the individual patient is exciting. It could be an incredible medical advance or it could prove to be impractical and end up a bust. In either event, to move forward intelligently, and to avoid being swindled, more control must be exerted over a testing process that has misled us so often in the past.
Clifton Leaf is the author of "The Truth in Small Doses: Why We’re Losing the War on Cancer — and How to Win It."