Pre-registration Of Clinical Trials For New Drugs Causes Them To Stop Working

[haidut]

The title is a sort of a joke of course. The real reason is that the drugs that are being tested are ineffective in the vast majority of cases (92%). Before public resgiration of clinical trials became mandatory, majority of the trials reported positive results. Now, think about all the drugs that you or people you know take on a regular basis and were approved based on trials prior to year 2000...

Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time
"...We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. 17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical trials.gov was strongly associated with the trend toward null findings."

Preregistration of clinical trials causes medicines to stop working! - Chris Blattman

 

[JohnCC]

This is interesting, but I can't quite get my head around it. What is the takeaway? What would one suppose to the the cause or explanation of it>

 

[haidut]

This is interesting, but I can't quite get my head around it. What is the takeaway? What would one suppose to the the cause or explanation of it>

I think the explanation that the blog post and some other outlets are offering is that the drugs that go through clinical trials are a lot LESS effective than we are being manipulated into believing. Once the requirement for public registration and publication was put in place, pretty much all subsequent trials with new drugs failed to show benefit. Unless you believe that something happened in 2000 that caused all drug companies to simultaneously start developing ineffective drugs, I don't see what other plausible explanation there may be. But I am always open to hearing novel interpretations, so please share if you have some.

 

[Koveras]

This is interesting, but I can't quite get my head around it. What is the takeaway? What would one suppose to the the cause or explanation of it>

There's less opportunity for them to manipulate the outcome variables, subjects, etc. in order to show a positive outcome.

 

[jaa]

This is interesting, but I can't quite get my head around it. What is the takeaway? What would one suppose to the the cause or explanation of it>

Say you run an experiment on Drug A to test it's efficacy for Condition 1. You know that if you're experiment shows positive results, there's a 99% chance those results are real and a 1% chance it's just noise. So if the results are positive, odds are Drug A works for Condition 1.

Now suppose you run another experiment on Drug B, except you don't set what you are testing for from the outlook. Say there are 100 Conditions you could possibly test for (Conditions 1-100). Like the previous example, if you were testing for one specific Condition, a positive hit would have a 99% chance of efficacy. But since you're not defining what you're testing for, the odds that you get a false hit any one of those Conditions 1 through 100 is much greater than 1%. It's > 50% too. My math is fuzzy and I don't have time to look up the actual percentage, but this is just to illustrate that false positives are much more frequent when you don't define what you're testing for beforehand. That's why this pre-registration thing came into effect.