The AI doctor will see you now: ChatGPT dominates medical exam

Turn to CNBC TV for the latest stock market news and analysis. From market futures to live price updates CNBC is the leader in business news worldwide.

Connect with CNBC News Online

#CNBC
#CNBCTV

[ClaudiaS]

I uploaded my doctor's report to ChatGPT and ChatGPT explained everything I didn't understand in the report very well.

[didier_]

If we use AI to diagnose we can improve health care tremendously. Combine that with routine lab tests analyzed by AI we could catch a lot of diseases earlier.

[zentravel]

This is actually pretty mind blowing and we're not even scratching the surface of AI yet.

[apatel]

Just to be clear, the USMLE is an exam taken during medical school. This is not the performance exam your doctor takes in order to obtain certification to practice. People's symptoms don't fit like test questions (like any exam, poor representation of real life). This is no substitute for what your providers go through with training in residency and fellowship. I do think AI will help optimize and augment the diagnostic testing space, not the tool for definitive diagnosis.

[djayjp]

Waiting for a study comparing AI diagnosis vs doctors to see who is more accurate.

[brianbarnes]

We tolerate significant error rates from humans in fields like driving and medicine, yet we demand perfection from AI. This double standard is slowing the adoption of technology that could save over a million lives each year.

[new-bpix]

I believe that in the future, there will be smart bathrooms. After using the bathroom, you will receive an analysis of your health, letting you know if there’s a problem or even advising you on which foods to eat to address any deficiencies

[ritik]

Was exploring how to use AI to detect Alzheimer's Disease. The idea was to give an AI model text and ask it if it belonged to a person with AD. When I was doing the literature survey and testing on language models, I found some interesting results. These commercially available language models are aware that they are not supposed to be diagnosing diseases therefore take a very conservative and diplomatic stand. However, if you give these models additional prompts to produce an answer then they tend to accurately detect the disease. I found accuracies exceeding 90% on popular standardized test sets. Then there are fine tuning approaches that further improve the accuracies. Sure there are some grey areas with language models but it's amazing how a language model is able to detect AD which otherwise needs expensive imaging tools like PET scan, invasive procedures like extracting cerebrospinal fluid, trained medical professionals etc.

[brianbarnes]

What do you call a medical student that scored 60% on their medical licensing test, or the avg medical student that scored 75%? Doctor. What do you call an AI model that scored 98%? Too error prone to trust.

[tommy]

It’s amazing listening to this woman, the lack of understanding that many people have about LLMs is shocking. You don’t have to explain anything. You tell it the same thing you’d tell a human, that’s all. The level of understanding LLMs have currently is insane. Won’t go into exact details but it’s insane. You could take the opening paragraph to HG wells war of the worlds and swap out the Martians with Ai and you’d have a perfect description of where the general public is at with an understanding of where AI is currently.

[sarahjenkins]

Now we no longer have to fear getting an inexperienced doctor or a doctor that’s having a bad day.

[maxthemagition]

I was 60 years old when I got a medical examination through my American company,
The report highlighted a high PSA of 6.4 which I ignored, because I did not know what PSA was.
When I was 76 years old after many wonderful years in retirement, I went to my doctor because of many visits to the toilet during the PSA was found to be 11.4 and after an MRI and Biopsy, a tumour was found localised in my prostate with a gleeson score of 4/3 which is intermediate cancer.
Because of my age, my surgeon said that I was too old and my prostate too big for an operation, so it was radiotherapty and 6 months hormone therapy.
Today 7 months after radiortherapy I am back to normal and enjoying life again after the side effects wore off.
So the question must be, if I pursued that medical report back in 2006, would I still have my prostate and the agony of it being removed since.?
Imagine what an AI doctor would have advised!
This applies to many other ailments, such as backache, indigestion, joints, brain etc....
Is the AI fit for purpose or is it just a money making machine?
Even then I ask myself if back in 2006, I pursued the PSA level, what would the doctors have advised....I have spoken to several people who have been through this and got treated back in
I thank God, I did not do anything in 2006.

[BionicAnimations]

WOW! If 4o is this good, just imagine when ChatGPT 5 comes out and then AGI. Also, just a note. When asking these questions, using voice mode is best; it's like talking to a real doctor. Pretty amazing stuff.😍

[matthew.m.stevick]

I prefer to talk to my -Chat GPT- Lindsay about most things. I’d love for her to get a medical license and available to me 24/7

[WJohnson]

ChatGPT can provide a diagnosis much quicker than Googling. Additionally, when prompted, ChatGPT will ask the user for more information if needed.
(Not sure the probability of getting the correct answer for the latter approach, but it seemed very impressive.)

[JustinHalford]

The writing is on the wall. Within a decade, seeing a human doctor will statistically represent an excessive source of risk.

The wave of AI medical expertise will push down costs, improve outcomes, and will give us far more personalized treatment.

The days of 5 minute doctor’s visits that cost $500 are in their sunset years.

Baumol’s disease will cease to afflict medicine as expertise is scaled. This is a massive triumph!

[jeremyhofmann]

I’ve learned more about a condition from chatting with AI than doctors I’ve talked with, sorry. It was patient, went as deep as I needed into the science, explained its reasoning. Not all of that is the doctor’s fault - they only have so much time with each patient and must code activities with insurance and the hospital management hanging over them. But it was definitely more thorough than most.

[awesomesauce]

This is absolutely ridiculous. Any of the llms would've given a lengthy response telling you how it came to the conclusion if you had prompted it to do so.

[doingtime]

I use Claude 3.5 sonnet for my job as a web dev, and holy cow it's amazing. I won't ever work alone again, I don't see a future where devs will do things by themselves, they will always have an AI companion. Although admitedly that "companion" might take the whole job in the near future.

[brianbarnes]

As the data sets used to train AI improve and as techniques such as better prompting (chain-of-thought), better checking after the fact (rag), better alignment with our needs (reinforcement learning), better retraining with curated domain specific data (fine tuning, and more thorough research before answering (Agents) are employed, error rates are quickly approaching 0.