ChatGPT vs. World's Hardest Exam

In this video, I discuss the solution to a problem from the 2022 International Mathematics Olympiad and see how ChatGPT fares with it.

⭐️ Links/references:
Solution to the IMO problem from ArtofProblemSolving:

Timestamps:
0:00 - The IMO Grand Challenge
2:33 - 2022 IMO Problem
8:18 - Giving it to ChatGPT
10:56 - A different model
13:43 - Patron Cat of the Day

[clawsoon]

I heard a junior high math teacher on the radio saying they got their students to figure out what was wrong with ChatGPT answers to math questions, and they said their students loved doing it. Nothing like the thrill of proving you're smarter than a computer.

[esotericheric]

You are a true teacher. The world needs more people like you. Please keep doing what your doing, it’s incredibly important.

[aasz]

I gave ChatGPT a math task from 4th grade of primary school. I asked the same task a few times. Each time, ChatGPT gave me a diffrent answer and every one was wrong. It took a few months for ChatGPT to correct its answer. This just shows that AI should be used as reference rather than a source of knowledge.

[SxC]

As a small tip, I found that I get better math and logic results from chatGPT when appending "Explain your resoning step by step." to the end of my question. It still gets things wrong, but it's more likely to be correct..

[MarinaMcPeak]

A really interesting video.

As you said Tibees, one of the limiting factor for ChatGPT here is its inability to parse the language used to describe the problem effectively. From a linguistic standpoint, I think mathematical problems are poorly worded even for humans to understand. We’re born with the ability to form language, but most people will never read and understand what these problems are even asking.

This is mostly because math problems rely on abstract concepts, symbols, and ideas that aren’t explicitly stated or consistent between problems (note that this isn’t limited to math problems). Furthermore, some of the concepts are poorly articulated in plain English. For example, defining infinity or what n-space is. So for a natural language model, it has no way to “reason” about these concepts if they aren’t tangible, explicit, and relatively consistent. This is assuming ChatGPT was trained on similar problems in the first place.

I see a challenge like this being equivalent to trying to get an AI to argue a philosophical concept. It’s incredibly difficult, if not impossible, for humans to understand and communicate abstract concepts with no miscommunication. So how could we expect AI to reason and communicate better than we do?

I’m sure one day AI will have the ability to solve these problems, but it’ll probably be because it’ll transcend the limitations of language and human reasoning.

[seandudley]

This is a lot, Thank you! I'm not sure how to feel about math getting even harder to test in, really. You've done such an amazing job of demistifying ai learning here though, Thank you again!

[diegovasquez]

ChatGPT is like the kid who’s really smart but didn’t attend the lectures

(Edit: I suppose it’s more like the kid who’s really smart, and has watched a lot of YouTube videos on the subject, but didn’t attend the lectures, so they have sort of a piecemeal understanding)

[nj]

I'm absolutely amazed by your ability to explain difficult maths problems in such an easily understandable way.

[Shilpithakur]

Thanks tibees for the upload it's wholesome how you explain things so detailed yet simple ☺️☺️

[StoietheWonderer]

You know what, i only watch your videos because of your soothing voice. Your shared knowledge is just a bonus.. your voice reminds me of my childhood, weird but it is nostalgic and relaxing

[NotAyFox]

This has probably been noted already, but you are, kind of, the Bob Ross of math.
I'm not very good at mathematics, but this video has been very engaging, approachable, and surprisingly relaxing to watch.

[markivory]

In terms of supporting Human creativity it is insane. I'm a Boatbuilder and left school at 15 and used it to script some programs that I knew I needed but couldn't write myself for work. Like anything you build 90% of what you do is how you hold the work, I know basic Linux Terminal from micro controllers. Feedback loop with GPT fixing it's own problems. It will tutor you on various aspects. Saved me near on 10 weeks of research for another project. We really need to talk about the future of work. Great Vid as always Toby : )

[Gustav.J]

I tried giving ChatGPT a physics question and the answer was completely absurd.

[rg]

I just discovered your channel and I subscribed right away. I love the way you explain maths (I learned a lot) and your voice is so ASMR… great job 👏🏼

[wilfreddsouza]

Ur voice is so soft and calm... I cant imagine how you would look screaming

[manticore]

I've given ChatGPT some very difficult engineering and physics problems and it's blown me away at how great it was at answering them, then I would give it something simple and it couldn't get it right.

[SacredSecret]

I have learned more just from watching this channel, and for me, it's always interesting. Like having a schoolmarm over the air. I'm very appreciative. Thank you.

[circusninja]

Much of what you teach is beyond my depth, but I could listen to your voice all day.

[ShakeyBox]

The idea that computers are better at art than I am, yet worse at mathematics, is quite amusing.

[neowy]

This is some high quality content. Highly informative and engaging. As an aside, also happens to be a great unintentional ASMR channel 😂