Using Healthcare-Specific LLM’s for Data Discovery from Patient Notes & Stories

Clinicians, confronted with a multitude of complex and pivotal decisions throughout a patient’s healthcare journey, often rely on population-level averages, intuition, and incomplete patient records.
Meanwhile, the electronic health record (EHR) is a rich repository of unnormalized clinical data, documentation, and patient attributes both structured and unstructured. EHRs are primarily focused on record-keeping and billing, but with AI we are maximizing the value of this data by transforming EHRs into a system of intelligence designed to enhance the care of patients and deliver a comprehensive understanding of each patient.

Electronic medical records (EHR’s) contain a lot of structured data and unstructured text about patients. It is most often messy, incomplete, inconsistent, duplicative, and as a result requires a lot of time from doctors and data professionals to get answers from. Large language model (LLM) generative AI interfaces can potentially improve the efficiency and completeness of these workflows by enabling providers or analysts to just ask natural language questions and get short, straight answers – but how accurate and reliable are they?
This session describes benchmarks and lessons learned from building such a pilot system on data from the US Department of Veterans Affairs, a health system which serves over 9 million veterans and their families. This collaboration with VA National Artificial Intelligence Institute (NAII), VA Innovations Unit (VAIU) and Office of Information Technology (OI&T) show that while out-of-the-box accuracy of current LLM’s on clinical notes is unacceptable, it can be significantly improved with pre-processing, for example by using John Snow Labs’ clinical text summarization models prior to feeding that as content to the LLM generative AI output. We will also review responsible and trustworthy AI practices that are critical to delivering these technology in a safe and secure manner.

Connect with us: