A surprising medical finding caught the eye of NPR's veteran science correspondent Richard Harris in 2014. A scientist from the drug company Amgen had reviewed the results of 53 studies that were originally thought to be highly promising — findings likely to lead to important new drugs. But when the Amgen scientist tried to replicate those promising results, in most cases he couldn't.
"He tried to reproduce them all," Harris tells Morning Editionhost David Greene. "And of those 53, he found he could only reproduce six."
That was "a real eye-opener," says Harris, whose new book Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billionsexplores the ways even some talented scientists go wrong — pushed by tight funding, competition and other constraints to move too quickly and sloppily to produce useful results.
"A lot of what everybody has reported about medical research in the last few years is actually wrong," Harris says. "It seemed right at the time but has not stood up to the test of time."
The impact of weak biomedical research can be especially devastating, Harris learned, as he talked to doctors and patients. And some prominent scientists he interviewed told him they agree that it's time to recognize the dysfunction in the system and fix it.
"If it's not operating at full steam ... and not doing everything right," Harris says, "it's worth pointing that out and saying, 'No. Think about this. Let's make it better.' "
The following has been edited for clarity.
Interview Highlights
On the ways unreliable research results affect patients
Tom Murphy was a healthy rugby player diagnosed with ALS in his 50s. .... With his doctor's help he signs up for an experimental treatment with a drug called dexpramipexole, or "Dex." At first, he's very hopeful, and it seems to be helping him, but they run the tests and figure out that it actually doesn't work. In fact none of the ALS drugs work. I focus on Tom Murphy because he's a victim of the system here — of these failures.
What happened in the case of ALS was there were at least a dozen drugs that had been tried in a handful of small studies — way too small — of animals. And they all seemed to have some sort of promise — some of them went into very large clinical trials. We spent tens of millions of dollars developing these drugs, and they all failed. There's a group in Cambridge, Mass. — the ALS Therapy Development Institute — that went back and reviewed all these studies and realized all the initial studies were wrong. They used very few mice. They weren't thinking enough about the different genetics of the mice. And a lot of other problems. ... This therapy institute came away thinking none of these drug candidates were really realistic.
On the ways the scientific enterprise in Charles Darwin's time was very different
Darwin was very interesting. It took him decades to come up with his theory of evolution and he was not in a hurry — he was studying barnacles, he was studying birds, all sorts of things. He felt no pressure to publish until somebody came up with a similar idea, and he decided, 'Hmmm ... maybe I do want to be first. ..." But we're not in that world anymore. Things are very competitive, very fast-paced. So the competitive world of biomedicine is shaping this problem of evidence that can't be replicated a lot.
On why the delight that's long been an intrinsic part of science can disappear over time — and why that's bad
I think a lot of people go into science out of a sense of wonder. But ... as time goes on, people feel the career pressures, and they realize it isn't just about exploring and having big ideas. They have to have research that helps them progress toward their first job, toward tenure, then the next grant, and so on. Those pressures are different from just, sort of, exploring and understanding fundamental biology.... And the less you're focusing on delight, the less maybe you're aiming at the truth and the more you are, inadvertently, often aiming at other goals — career goals, financial goals and so on. This may give you a fruitful life as an individual, but may produce less value to us as a society.
On how the public should respond when they hear of a big biomedical advance
I think it is good to question it. Every time you hear something like this, just remember, it's all contingent — here is one study, and it may not stand the test of time. I think that's healthy. ... When scientists read the scientific literature, they realize, "Oh, probably half of this is wrong." It's just, not knowing which half — that's the vexing part.
On the risk that pointing out flaws in science will make people question its value
It's always uncomfortable to point out problems, but it's also essential. I mean, we are taxpayers — we are citizens, and we support this enterprise and we expect to reap its rewards. If it's not operating at full steam ... and not doing everything right, it's worth pointing that out and saying, "No. Think about this. Let's make it better." Many prominent scientists agree with me and are concerned about this — and are thinking hard about how to make things better, from the top of NIH on down. There are solutions, and I talk about them in my book.
On why the Trump administration's proposed cuts to NIH funding wouldn't make things better
It's a very appealing idea, obviously, to say, "Oh, well, let's just identify the waste and root it out." But that's not the way science works. ... If you cut the [$30 billion] budget of the National Institutes of Health, you're going to shrink that already very small pool of money even smaller, and you're going to increase the competitive pressures. You're going to increase all these perverse incentives that put us in this position to begin with. So I think that would actually be devastating to biomedical research.
Copyright 2020 NPR. To see more, visit https://www.npr.org.