Health Sciences and the Accuracy Crisis

 A recent issue of the London Review of Books provided an article by John Whitfield titled Replication Crisis.  It reviewed the book Science Fictions: Exposing Fraud, Bias, Negligence, and Hype in Science by Stuart Ritchie.  In this pandemic/climate change era when we are constantly demanding that people “follow the science,” we should be concerned about a book with such a title.

The replication crisis in Whitfield’s title refers to the fact that scientists in critical fields are churning out papers reporting scientific findings that other researchers are unable to reproduce.  If the results are not reproducible, how can they be considered accurate?  And what if public or medical care policy is based on a result later found to be incorrect?

Most of these studies are carried out by academics and funded by government agencies.  “Publish or perish” is the order of the day.  The pressure begins as soon as a graduate student enters a doctoral track.  The student must demonstrate the ability to produce publishable research to obtain a doctoral degree, and his/her professor must demonstrate the ability to produce students capable of producing such work.  Getting results published in a prestigious journal is how academics gain tenure at a university and is key in obtaining funding to continue to perform research.  The new post-graduate student is under continuing pressure as he/she navigates the academic world looking for employment which will hopefully lead to a permanent position somewhere.  Note that professors can produce many doctoral students over their careers, many more than the system can absorb.  The competition is intense and never really ends.   

“In Science Fictions, Stuart Ritchie explores the problems with this system. The book is a useful account of ten years or more of debate, mostly in specialist circles, about reproducibility: the principle that one purpose of a scientific paper is to make it possible for others to carry out the same work, and that one test of its reliability is whether they get the same result.”

“In recent decades there have been large-scale efforts at replication in several fields, but if an experiment can’t be repeated, it doesn’t necessarily mean the original work was incompetent. Work at the frontier of a discipline is difficult, and skilled hands are an underacknowledged factor in scientific success… Even so, the findings of these large-scale replication studies have helped to fuel a widespread sense that science is failing on its own terms: in cancer biology, one effort managed to replicate just six out of 53 studies; in psychology about 50 per cent of studies cannot be replicated; in economics, about 40 per cent. In 2016 Nature surveyed researchers across the natural sciences and found that more than half the respondents had been unable to repeat their own work, though less than a third regarded the failure as a sure sign that a study was wrong.”

This system demanding continual proof of accomplishment via journal publications provides dangerously perverse incentives at numerous points.

“At one end of the replication crisis, as it has become known, there are spectacular frauds. In the early 2000s the South Korean biologist Hwang Woo-suk became a national hero for cloning human stem cells; just a few years earlier, the materials scientist Jan Hendrik Schön was being tipped for a Nobel Prize for papers describing molecular-scale electronic components. Both had made up their results. In surveys, about 2 per cent of researchers admit to fabricating data, though many more suspect their colleagues of doing so. But deliberate malpractice probably accounts for only a small portion of unreliable science. The greater concern is that the rush to publish and the pressure to make a splash pushes researchers to take short cuts and dodges: low-level fiddles that stop short of fraud but undermine reliability.”

“The worry is that scientific processes have been undermined by perverse incentives to the point that it’s difficult to know what to believe. The crisis has hit psychology, Ritchie’s own discipline, and biomedicine especially hard. These are crowded, competitive fields, in which research groups around the world are racing one another to publish on the hottest topics. In these circumstances, haste can win out over care. The data in these fields tends to be noisy, leaving room for interpretation and manipulation in presentation and analysis, and psychologists and biologists tend to be less mathematically expert than their colleagues in the physical sciences.”

There are also financial incentives driving the need to publish.  Professors’ salaries are generally merit based with the proof of merit demonstrated by their publication lists.  In addition, there are opportunities for more direct forms of compensation.

“There can be financial incentives too…many Chinese universities were giving cash bonuses for publications, with higher-impact journals securing bigger rewards for researchers. In a survey of Chinese university policy in 2016, the average bonus for the lead author of a paper in Nature or Science was calculated at $44,000, five times the average professorial salary.”

In no medical arena is the opportunity for and the realization of false results more egregious than in the pharmaceutical industry where biased research and unethical and illegal activities have become common business practices.  These activities are not new developments.  About ten years ago they were discussed in the articles Lies, Damned Lies, and Medical Science, Medical Science and the Vanishing Truth, and Drug Companies: Even More Corrupt Than Financial Institutions?.  A recent article by Brian Buntz, GSK, Pfizer and J&J among the most-fined drug companies, according to study, indicates that these practices continue.  Rather than send criminals to jail, large, powerful drug companies get their hands slapped by merely assessing them negligible fines (compared to the profits gained) if they promise to behave better in the future. 

“GlaxoSmithKline (LON: GSK) paid nearly $10 billion in inflation-adjusted financial penalties between January 2003 and December 2016, the highest tally for any drug company, according to research published in JAMA.”

“Pfizer (NYSE: PFE) was next in line with almost $3 billion in fines.”

“Johnson & Johnson (NYSE: JNJ) came in the third slot with $2.7 billion in penalties.”

“In all, the 26 pharmaceutical companies paid some $33 billion in fines during the 13-year period. The top 11 alone accounted for $28.8 billion, or 88%, of the total.”

“’The pharmaceutical industry is unique in that all large pharmaceutical firms explicitly state that they are focused on promoting patient welfare, yet the majority of large pharmaceutical firms engage in illegal activities that harm patient welfare,’ said Denis G. Arnold, a coauthor of the study and a professor at the University of North Carolina at Charlotte.”

It is interesting, and perhaps a bit scary, that two of the companies on which we have depended for Covid vaccines are among the worst offenders.

Scientific research can produce extraordinary results, but shoddy and inaccurate findings can undermine the general trust in science.  Between Covid and climate change, we are in a position where we must follow the science and the science must be correct.