The strengths and limitations of randomized controlled trials

It’s important (I think) to understand where RCTs are helpful—essential, really—and where they are ill-equipped to answer questions. In this video I lay out the basics but the mega question is will AI change this paradigm? Will simulation change this? Or merely speed it up slightly? I plan to do another podcast on AI in medicine and would love to tackle this question. The cost of RCTs and their very narrow scope make this a wild opportunity.

--------
About:

The Peter Attia Drive is a deep-dive podcast focusing on maximizing longevity, and all that goes into that from physical to cognitive to emotional health. With over 90 million episodes downloaded, it features topics including exercise, nutritional biochemistry, cardiovascular disease, Alzheimer’s disease, cancer, mental health, and much more.

Peter Attia is the founder of Early Medical, a medical practice that applies the principles of Medicine 3.0 to patients with the goal of lengthening their lifespan and simultaneously improving their healthspan.

Connect with Peter on:

Subscribe to The Drive:

[KeyserTheRedBeard]

Impressive video, Peter Attia MD. Looking forward to seeing your next upload from you. I hit the thumbs up button on your content. Keep up the fantastic work! Your insights into the limitations of randomized control trials are crucial. How do you envision integrating emerging biomarkers into clinical practice to enhance our understanding of age-related diseases?

[jaymckoskey]

What about N=1 experiments? They can offer useful insights with high control and phenotype specificity. I would think that a metanalysis of a thousand such studies would be powerful.

[HeatherQ]

Well, let's go already. I heard a lot of limitations, but no strengths. Such is the mind of a brilliant and conditioned academic... ? I'm thinking about getting in to research because I feel like a lot of time, resources, (etcetera) is being wasted, and opportunities and out of the box thinking underutilized. Not you specifically, Peter. In general. Thanks. 🙂❤

[davidferry]

Blinding is desirable but not necessary to draw clear inferences. Take the time when a belief that high dose chemo in adjuvant breast cancer was beneficial took hold. Many argued it was unethical etc not to randomize against low dose chemo the standard of care. Three RCTs, not blinded showed huge harm done by high dose chemo, a huge shock to the majority of believers. Thus although RCTs are often not perfect in every way they allow robust inferences and do so much good. Biomarkers are always going to be surrogates and the false belief they can be discovered without RCTs will be enormously harmful to medical progress.

[yunggolem]

How did they come up with the idea it's "not ethical" to study people who are already going to do the thing that's bad for them? For instance, smokers. What are the mental gymnastics that make it unethical to study people who are already smokers?

Sure if you encouraged them to be smokers or continue smoking, which would provide less noisy data, but you'd need to contort your ethical obligations outside of feasibility to make a valid claim that you have a duty to intervene & stop someone from harming themselves from smoking & therefore it is unethical to even OBSERVE a smoker smoke & measure the effects. Accepting that idea as a principle & acting consistently with it would turn you into the nanny of everyone in the world & one of the worst people on earth.

If anything, by measuring the harm a vice does & sharing that information, you ARE intervening in a way that could help them at a deep psychological level, rather than the brute force chemical methods often used which fail very often.

The argument that this kind of study using existing vice indulgers is "unethical" is so paper thin I have trouble coming up with one that doesn't sound ridiculous. Perhaps it is more to do with legal liability than actual ethics... or maybe some other factor.

[Dr_Boult]

You did not really cover the strengths which mighe be useful other to understand. For limitations agree with 3 out of 5. Binary is definitely not a requirement of RCT, and even a pill is not really binary. Many good RCTs have used dosage as a part of the design and dose-dependent response curves can themselves be very useful in helping to understand causality. Blindedness is not required, though I agree its helpful. If the effect size is so small that a placebo effect, and hence blinding, is of comparable size then one has a pretty minimal intervention. But I think you also missed what I consider a (or the) major limitation of RCT for causality testing -- the huge variation in human responses makes it only a partial indicator -- if an intervention has an outcome for a fraction of the population but not for all, then the RCT cannot separate if the intervention was the true cause, or just a contributing factor where something else is causal but the impact/effect size is impacted by the intervention. Something that is necessary, but not sufficient is not causal, even if it may be in the causal chain of events. Protein is necessary for the development of arterial plaque, that does not make it causal.