The medical test paradox, and redesigning Bayes' rule

About Likelihood Ratios, also sometimes called Bayes Factors*.
An equally valuable form of support is to simply share some of the videos.

The book by my friend Matt Cook about paradoxes mentioned at the end:

On the topic, I can't help also mentioning another paradox book I'm rather fond of by Bunch:

Another video on Bayes' theorem:

*As mentioned in the on-screen note at the end, while the terms "Bayes Factor" and "Likelihood Ratio" refer to the same ratio in this setting, where Bayes rule is used on the probability of an event with only two possible outcomes (you either have the disease or you don't), they do take on divergent meanings in more general contexts. Namely, if you have a continuous parameter you are trying to estimate, the two terms reflect two alternate approaches you can use in comparing hypotheses. In fact, some people take the phrase "Bayes factor" to _specifically_ refer to its use in this more continuous context.

If you want more details, Wikipedia actually has a really nice example discussing the difference:

This post has some nice discussion of the distinction:

Timetable:
0:00 - What is the paradox?
5:53 - The Bayes factor
11:00 - The snazzy Bayes rule
14:42 - Contrast with the usual formula

Thanks to these viewers for their contributions to translations
French: Raphaël Ventura
Russian: Ilia Konnov
Vietnamese: duyhub

------------------


If you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and it has many other quirks you might expect in a library someone wrote with only their own use in mind.

Music by Vincent Rubinetti.
Download the music on Bandcamp:

Stream the music on Spotify:

If you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then "add subtitles/cc". I really appreciate those who do this, as it helps make the lessons accessible to more people.

------------------


Various social media stuffs:

[johnchessant]

This is awesome. I have now updated my odds of correctly answering a Bayesian probability question.

[connermcd]

As a doctor I'm so happy you're using your platform to get this information out. Let me tell you though... it gets way more complicated! Unfortunately prevalence estimates aren't always known and are constantly changing (especially in pandemics). Another thing to consider is the gold standard. If your test looks for breast cancer you can cut out the lump and look at it under a microscope. Some diseases aren't as easily clarified. For instance, since we don't have a highly accurate, easy test for pancreatic cancer we rely on imaging, demographics, blood markers, symptoms (or lack thereof) as multiple things that form a conglomerate test to increase our Bayes factor. Despite all these things we can't always get a great prediction on whether that scar in your bile duct is cancer or just a residual scar from pancreatitis you had 10 years ago. So we offer the patient a huge surgery to remove the head of their pancreas and duodenum only to find that it wasn't cancer. You can imagine the patient is happy it's not cancer but not so happy they don't have half their pancreas and have abdominal pain and maybe diabetes. Medicine is a tricky thing. Another tricky thing is operator error. Some tests depend on the skill of the lab tech, radiologist, or surgeon. The complexity of the human body and the uniqueness of each individual also plays a role. Your test may be false positive in a particular patient 100% of the time because they have some strange protein mutation. It's tough!

[Asdfgfdmn]

I hope Grant read this 😇

I am an MD and Associate Professor of internal medicine. I teach medical students, residents, and fellows. I used to be a program director for a fellowship at a prestigious American university. This is a recurring lesson I teach. The example I usually use is the DNAJB9 kidney biopsy stain sensitivity and specificity for a disease called Fibrillary GN., and I do the exact walkthrough with my students. I never get bored when I see how surprised they are with the final conclusion. Which is, by the way, is: you can't use a test willy-nilly without considering the pre-probability (you are referring to it here as ”prior.” And I also tell my students that you can increase the prevalence of the disease is by applying it to the right population (signs and symptoms). I am thrilled that Grant validated this with this awesome video

[Dn]

Medical Student here, THIS IS GOLD. THANK YOU, this is going to help with my boards and future patients

[Jebusankel]

I'm going to apply this to the world of dating. Everything I learn about a potential match updates my prior about our compatibility. I call this Bae's rule.

[kingbradley]

This is actual gold content being uploaded for free. It's like I'm unlearning what I learnt in all my classes and seeing Maths in a whole new way. I was asked in an interview one concept that is often confused but makes sense in general. I spoke about Bayes' Theorem. And this has given me something more to talk about. Quite possibly the best educational channel on YouTube.

[hjfreyer]

This is the first presentation of Bayes' theorem that didn't leave me feeling both like it was trivial and like it was inscrutable magic.

[DJNHmusic]

As a young doctor, thank you so much. I understood the distinction between the different accuracy parameters and PPV beforehand, but this has fundamentally changed how I view testing. This is a very useful thing to understand as a medical professional.

[KX]

As a scientist who does medical tests, I'm amazed that any of the doctors asked got the right answer. Every doctor I work with assumes tests are 100% accurate.

[RasperHelpdesk]

Certainly drives home the point of why running a test twice after getting a positive result is so important when possible.

[tejing]

This perspective makes the 'update' concept so much cleaner. I've long believed that until something is utterly obvious to you, you still don't truly understand it. I just got much closer to understanding bayesian updating. I could already do it, and even explain it, but it wasn't the same. True understanding is precious. Thank you for what you do, and as always, I look forward to the next video.

[egegorgun]

Hey Grant, I am a third semester bachelor's student in physics and I find your videos very intuitive and absolutely inspiring. I find it very hard to deal with spherical harmonics this semester and as they are algebraicly complex but easy to visualize I thought maybe you can make a video about it too. It would help me very much at least. I can imagine these videos take a lot of effort and so I appriciate it very much. Thank You.

[wernerviehhauser]

This topic always breaks the brains of my students. Every year. Every time. This IS pretty hard stuff to wrap your mind around.

[davidholle]

what really helped me was not using words like "sensitivity" but instead "true positive"

[avalon]

Often when I see a video from you pop up in my abo box, I hesitate to click on it, because almost every time I get so involved that a say 20 minute video consumes, like, 1 hour of my time to really get through all of it and to understand it and to really get it into my mind to apply it in an every day fashion. But in hinsight, you NEVER fail to make the invested time worth it. Thanks for that.

[eyabs]

Thank you so much for this video. This concept of updating odds is what finally got Bayes'
Rule to finally click in my head. I've used it a million times at work and it always bothered me how I couldn't do these simple calculations without pen and paper.

[egillandersson]

As doctors, we use this every day, often without thinking about the mathematical foundations.
Unfortunately, very few diagnostic tests ou exams are indeed both sensitive AND specific.
So, if we think a diagnostic unlikely (based on prevalence, physical exam, previous tests, ...), we choose first the more sensitive test in order to exclude this diagnostic.
On the contrary, if a diagnostic is very probable, we choose first a specific test to confirm.
It is not always easy for technical exams, as we can often only choose between them (if there are several !), without changing their sensitivity and specificity. But for biological tests, we can adjust our cutting values to improve either sensitivity or specificity.

[aytide]

Grant never misses. He's always brilliant

[LoscoX]

I started watch your channel a month ago, and now you're my favourite math channel!

You talk about math in a not too formal and technical way, but at the same time, speaking about technical math things which encourage and help me to go on study math at the university.

Thanks for each your videos!

[cliffcrosland]

My favorite YouTube video of the year. I have already used this like a dozen times. Even used it to explain to someone how my personal beliefs evolved over time as new evidence updated my prior odds. Love it thank you!