ChatGPT Fails Basic Logic but Now Has Vision, Wins at Chess and Prompts a Masterpiece

ChatGPT will now have vision, but can it do basic logic? I cover the latest news - including GPT Chess! - as well as go through almost a dozen papers and how they relate to the central question of LLM logic and rationality. Starring the Reversal Curse and featuring conversations with two of the authors at the heart of it all.

I also get to a DALL-E 3 vs Midjourney comparison, MuZero, MathGLM, Situational Awareness and much more!

[hermestrismegistus]

By far the best AI-focused channel I've watched. AI Explained actually understands AI and its strengths/limitations rather than spouting unjustified hype or pessimism.

[DerSwaggeryD]

that was a hell of a naming fail. I first thought "V" meant 5. 😂😂

[GroteBosaap]

Love how you cover papers, contrast them with others, do your own testing, and talk about timelines to AGI.

[djayjp]

Fascinating insights, thank you. I love that you don't just report the news, but you provide deep insights into the state of AI.

[JL]

I also do actually think that this reasoning issue is actually very human like in a way. I've been learning Chinese for a while and here the learning gets even weirder : there is pronunciation of a symbol, being able to recognize it, being able to write it, knowing it's meaning and knowing the meaning of the whole word. Which means that sometimes I can understand a word, but not spell it. Sometimes I can spell it, but no idea what it means, sometimes I even know what the symbols mean but the whole word is too abstract and I can't remember. On some characters I can spot mistakes in their writing, but I wouldn't be able to draw them completely from my head.

The difference to chatgpt seems to be, that for humans this does not so much apply for higher knowledge about facts. So maybe the model learns "Biden is president" the same way as you learn "an apple is..." (enter your memories and feelings and sensations of an apple which cannot be described in words) and not like we do, where "Biden is president" is more like digital knowledge that sits on top of analog knowledge like what Biden looks like, what you associate being a president with etc.

[chiaracoetzee]

As someone who's done a lot of foreign language learning and had to memorize a lot of vocabulary, it's not unusual at all to know that e.g. "horse" and "cheval" mean the same thing but only be able to perform recall in one direction and not the other direction, unless I'm given some kind of hint to narrow it down. This is often called a "tip of my tongue" feeling when you know it but can't quite recall it given the current context. In that sense, this logical limitation might make LLMs even more human-like.

[dextersjab]

Super quick work with the video! It was crazy seeeing all this stuff unfold on Twitter. I still think there's plenty more experimenting to do with reasoning. We've clearly still got a lot to learn about intelligence.

[tiagotiagot]

Humans sometimes also don't remember things both ways with the same level of difficulty. For a simple (though somewhat weaker) example, ask someone to list the letters of the alphabet backwards, and then ask them some other time to list it the normal order; it's probably common sense that the vast majority of people will have more difficulty doing it backwards.

[TMtheScratcher]

Good points you bring up in the end: Our brain consists also of specialized "modules" - and our language processing capabilities alone do not handle our skill and knowledge around logic.

[Drone]

Excellent video. LLMs are making it obvious that many of the things we think of as logical thought are often pattern matching. To me this explains why some people come to an incorrect conclusion but so strongly believe it. They're pattern matching with insufficient examples like an LLM but they have no awareness this is what they are doing.

[TheBlackClockOfTime]

I almost had a heart attack when you said "GPT-V"

[tehlaser]

The fact that humans can do this is actually kind of amazing. By “this” I don’t mean realize that A is B implies B is A. That’s just logic, and GPT seems to be able to do that too, so long as you activate A and B in the same context.

The amazing bit is that, when we humans learn that A is B (or even that A is similar to B) we form associations in both directions. Thinking about A suggests B, and thinking of B suggests A. Most animals don’t form that reverse association. Humans do.

[oscarmoxon]

This is one of your most impressive videos yet. Since starting watching AI Explained, I've flipped from an Economics Bachelors to a Masters student in Artificial Intelligence at KCL, where this has become my full-time focus -- and I am privileged to witness your journey into becoming an incredible AI researcher along the way. The micro and macro you paint here are truly cutting edge perspectives. Always blown away by these vids.

The other day I spoke with Mustafa Suleyman at CogX about recursive self-improvement and multi-agent interaction. Curious what you think about this space and what you think of recent news here.

[mickmickymick]

This might explain why I very rarely get useful results from ChatGPT or Bing, even when I know it should have the information I want.

[LeonardVolner]

Once upon a time, I worked at a plant nursery handwatering plants for many hours every day. It was a task that left lots of freedom to think freely.

I taught myself the alphabet backwards...

It's nice to have a job so undemanding you can think about whatever you want but the caveat is that you don't have hands to take notes or research new material. It's somewhat limiting unless you can think creatively about how to use that thinking time prodcutively given the implicit handicaps.

[wndache]

Funny thing that as we are trying to teach LLMs generalised logic and reasoning, looking back at how we would think through puzzles, it's also filled with "cheap" rote learning, memorisation, and applying patterns...

[a.thales]

After the announcement of dalle3 i said to myself, that openai really should release this year a function to also upload pictures to talk about them, and to get the answers als voiced, just like we can speak into chatgpt via android and iphone... voilla! what a time to be alive. I really thought that we'll get these features this year, but i thought maybe december and not october! great. thanks for the video! i learned the news from the video and was like... how did i miss this? will watch the video now!

[danielxmiller]

This happened with me when I learned my states and capitals, I could tell you the state if you gave me the capital, but if you gave me the state, I, for the life of me, couldn't give you the capital. Looks like it's more like a mind than a machine!

[keeganpenney]

The definition part you brought up is even more brilliant discovery, it basically says we're all in an infinite loop in terms of logic, reasoning and rational that only ends when said party is satisfied with the return, whether it's correct or not actually doesn't have much to do with anything.

Bravo, seriously that's some next teir human falicy discovery right there.

[minhuang]

Full-on nailed the German pronunciation, very small sample size, but that sounded better than most learners