Can AI outperform Stanford Medical Students? (My own research!)

A group from Stanford gave ChatGPT free response, clinical reasoning final exams and compared its performance to students at Stanford medical school. How did it do? Featuring a discussion with co-authors Alicia DiGiammarino, Jason Hom, and Jonathan Chen.

The tl;dr (i.e. the results): @8:20

Discussion with my co-authors: @9:24

#ChatGPT #MedEd

[Steven-csyc]

Hi Dr. Strong, I am a new IM intern. I have been a long-term viewer of your channel, big fan. I appreciate that you post this video because ChatGPT and GPT4 have been on top of my mind for the past few months. I have used them in chart reviews and they have achieved remarkable accuracy. With the pace of this technological advancement, I am worried about the future for internal medicine doctors as AIs are becoming better and better at diagnosing diseases. While I believe that patients do value an empathetic human doctor, I wonder who they will choose when Dr. GPT can see them much sooner, reach the same diagnosis and sound more empathetic

[bakercat]

This is a brilliant and thought provoking conversation!

[liam_hurlburt]

fantastic discussion, thanks for providing a bit of background to the research with this video. I'm definitely fascinated to see where we end up in 10 years with all this, though as someone who is currently just a premed student I'd be lying if I said I wasn't worried both for my own future and the future of medicine!

[nickfosterxx]

Late to this, but just wondering how much better other models might have performed, and how well the current 'paid for' or Pro models would be.
Thank you, fascinating topic. This seems like an incredibly awkward transitional period (for educational assessment at least!) before the complete 'normalisation' of AI into diagnoses?
Absolutely blown away by some of your anecdotes eg ethical dilemmas 'in a safe space'.

[alvaronf]

Dr Strong, thank you for what you do for education in medicine ! I would really like a video of your opinion regarding the USMLE, for example, is it really important to evaluate knowledge on ultra rare diseases such as metachromatic leukodistrophy?

[shaukatmehmood]

Our mcqs exam is actually testing ur speed of reading, ur speed of answering...bt speed is not need of patients it's need of exam taking authority....the only need of patient is accurate diagnosis....if u reach to the accurate diagnosis in one hour time its fine for patients bt if u in hurry, in speed in minute choose incorrect diagnosis it's can be danger for patient

[SK-ivkc]

Hello Eric, I would some more details on how the formulation of the prompt influences the answer within medical context.

[tysken]

How would you say open AI o1 is compared to chat gpt 4 in clinical reasoning?

[WalidBourahli-cp]

Hello Doctor, please, I have a question about beta-blockers. You said that they reduce contractility in the short term and increase contractility in chronic treatment. Is this only in the case of heart failure? Because it controls angina pectoris and other diseases due to its reductive ability on the heart. It increases contractility in chronic treatment. How do you control angina in the long term? Please reply doctor, thank you

[samanthaperez]

You used to provide excellent, in-depth educational content and now you provide content that appeals on a broader scale for the YouTube algorithm. Maybe make a separate channel for this? I've seen other channels go down this road (e.g. medcram) and it really is terrible for those looking for medical content. Just my opinion. It's your channel, but please reconsider another channel for attempts at making viral content.