Why Large Language Models Hallucinate

Large language models (LLMs) like chatGPT can generate authoritative-sounding prose on many topics and domains, they are also prone to just "make stuff up". Literally plausible sounding nonsense! In this video, Martin Keen explains the different types of "LLMs hallucinations", why they happen, and ends with recommending steps that you, as a LLM user, can take to minimize their occurrence.

#AI #Software #Dev #lightboard #IBM #MartinKeen #llm

[illogicmath]

Finally someone who speaks at a human speed and not like those youtubers who over-optimize the audio by cutting all the pauses and even increasing the speed of speech.

[bestlyhub]

Summary of this video " Large language models (LLMs) can generate fluent and coherent text on various topics and domains, but they are also prone to hallucinations or generating plausible sounding nonsense. This can range from minor inconsistencies to completely fabricated or contradictory statements. The causes of hallucinations are related to data quality, generation methods and objectives, and input context. To reduce hallucinations, users can provide clear and specific prompts, use active mitigation strategies, and employ multi-shot prompting. By understanding the causes and employing strategies, users can harness the true potential of LLMs and reduce hallucinations. "

[urbandecay]

They hallucinate because they have taken LLSD : Large Language Standard Deviation.

[cyndicorinne]

The first video I’ve seen that gives a detailed outline of the problem of LLM hallucinations and offers potential solutions or at least mitigation techniques. I love this 💜

[manomancan]

I really appreciate this work, thank you! Always great when IBM's channel produces a video like this. Really like the presenter too.

[citizen_of_earth_]

Hallucinations remind me of Jimmy Kimmel's segment "Lie Witness News" when he asks random people on the street about events that didn't happen. They usually make stuff up that sounds plausible. LLM's seem to be doing the same thing.

[MrHaggyy]

Hallucination is a great way of thinking about these problems that was new to me. Thanks IBM for sharing this, and also great work in building that picture to guide the talking.
I experimented with these effects by asking/prompting LLM's about a book i know very well, that is discussed a lot online and might even be available in the training data. Things like wiki books about math or programming, or Why We Sleep by Matthew Walker.
It was shocking how far of the real contend a broad question could be. But it was also interesting how good these models can cite/copy the original if get very specific and don't leave it with a lot of options. I always thought of it as an alignment problem and how guardrails in ChatGPT and BingChat prevent it from basically printing entire books.

[evanmacmillan]

this channel is like a treasure, the topics are so interesting and useful, and teaching in an easy call yet very enjoyable to watch. I'm addicted!

[seanendapower]

I do wish the term ‘hallucination’ was not the term. There’s a perfectly good term ‘confabulation’ right and implies what we actually experience this phenomenon as and also what we know is going on. ‘Hallucination’ is a significant element of perceptual psychology tied to the hallucinator’s psychology, consciousness, phenomenology, epistemology, pathology … none of which we know to be applicable to AI without assuming it is a subject, a perceiver, a conscious experiencer, etc

[theelmonk]

It's not hallucinating. It's doing exactly the same with sensible output as with crazy output : making text that fits spelling and grammar rules and meets a word correlation policy known to match common articles. There isn't any sense in it beyond what we look for. You could just as well ask why LLMs talk sense : it's equally common and either sense or nonsense is just as acceptable to the model.
However, confirmation bias causes us to consider most of the output OK until we're faced with something accidentally so bizarre that we notice.
Put another way - subject-wise, hallucination and insight are oppsite ends of the bell curve. The vast majority in the middle is filler text but we call it sense because it's not obviously wrong, and we EXPECT it to be sense, so we parse it into sense.

[BarryKort]

Geoffrey Hinton suggests that the more correct term is 'Confabulation'.

[kappamaki]

asking about a video games lore is a perfect way to make LLMs hallucinate pretty much everything they say, if you ask specific enough questions or the game isnt terribly popular

[bestlyhub]

👀🤯 This video on hallucinating large language models is fascinating! It's amazing how AI has advanced so much that language models can generate text that's almost indistinguishable from what a human would write. The potential applications of these models are incredible, but it's important to consider the ethical implications as well. I look forward to learning more about this exciting field of research! 🌟. Thanks IBM

[muppetjedisparklefeet]

I spent 20 mins with Bard telling me about a public health program and answering questions about eligibility criteria, when it began and ended, and studies of the program’s outcomes on various health conditions (complete with doi links)- all made up. When I called it out I said it is learning and sometimes gets facts wrong. it was a trip.

[ILsupereroe]

If you thought writing backwards was a useless skill, think again😅

[quantumastrologer]

Having worked for over a decade with people who suffer from dementia and other various mental ailments I'm super glad that the skill to parse the patients mental output and filter out 'nonsense' (it always makes sense from the perspective of the patient) neatly transferred over to me trying to get a grasp on software engineering.

[ewallt]

What I’ve found is that it’s almost impossible to get what you want the first time, if it’s complex. You have to do it iteratively. However, once you have what you want, you can give that as a prompt, and tell the AI you want something like what you provide, and that works well.

[CorbinSimpson]

This is a myopic approach to LLMs which embraces the LLM-as-conversant view, rather than the more general LLM-as-simulator view. For example, the multiverse principle is not really mentioned; your cats-speaking-English example could have been enriched by explaining how an LLM conditionalizes its responses based on the possible logical worlds. Ultimately, the answer to the topic question was meandering and wrong: LLMs hallucinate because they can; emitters of words are not required to emit truthful sentences. (And Tarski proved that there's no easy fix; you can't just add a magic truth filter.)

[KevinMeyer]

Love Martin's videos - especially as they relate to beer in his Brulosophy channel!

[hwzhqey]

I'll agree on the data quality being a potential cause. Training methodology can also lead to unexpected outcomes. However, the core cause of hallucinations is really that the model hasn't properly converged in n-dimensional space primarily due to a lack of sufficient training data. The surface area of the problem being modeled increases significantly as you increase the dimensionality, meaning that you need a corresponding increase in the size of the training data in order to have enough coverage so that you have a high degree of confidence that the converged model approximates the actual target. These gaps in the spacial coverage leaves the model open to just guessing what the correct answer is leading to the model just making something up or hallucinating.