LLM hallucinations discover new math solutions!? | FunSearch explained

The unreasonable effectiveness of guided confabulation: Solving math problems with hallucinatory LLMs is now possible!? 🤯 We explain how Google DeepMind did it.
Bonus: The answer of Fields Medalist 2022 Hugo Duminil-Copin at the last #HLF23 after I asked about AI helping mathematicians solve problems.

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, @Mutual_Information , Kshitij

Outline:
00:00 Mathematics will fall first
01:19 time computation / Guided confabulation
03:02 FunSearch explained
05:14 Cap Set problem
06:23 Bin Packing problem
07:22 Mathematicians say *this*
10:42 Will AI take mathematician’s jobs?

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

🔗 Links:

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​
Mushroom in the thumbnail generated with DALL-E (GPT-4 interface)

Video editing: Nils Trost

[terjeoseberg]

This is how humans solve problems. We come up with a million dumb ideas, think a bit, and discard the 99.999% that are obviously wrong, and then verify the ones that aren’t. Every once in a while, we find a gem.

[tahir]

thanks for this excellent explanation. exciting times!

[opusdei]

Wow Letitia, your videos are amazing :) thank you

[ceilingfun]

I'm not a mathematician, but I love mathematics. Wolfram Alpha did not make mathematics obsolete, and in my opinion, nothing will.

[ricowallaby]

Love mathematics and science. . . I remember my maths teacher when we worried so much about how solving problems, and he said "remember every problem contains its own solutions". How true. Cheers from Sydney
'

[sanjose]

I really like your information delivery style ✨

[CaptTerrific]

Whoa... so they essentially ran a genetic learning algorithm with LLMs? That's rather clever (if not very very expensive :D)

[CodexPermutatio]

Hallucinations are not bugs... They are features! :]

[vitalyl]

I keep saying that hallucinations are the best feature of the LLMs. They are exactly the last missing piece of puzzle I needed to automate discovery and invention.

[TemporalOnline]

Almost like letting a million monkeys bashing a million keyboards for a million years and then checking if any of them wrote Shakespeare ¯\_(ツ)_/¯

[homewardboundphotos]

So... if we are back to brute force, what could be the implications of IBM's quantum roadmap they just released? I said right from the start, the hallucinations aren't a bug they are a feature. It's not a coincidence people who think very abstractly are more likely to get their wires crossed about things. You're never going to come up with a completely novel good idea without trying out 1000 ideas that don't work in the process. That's pretty much a law of nature in human intelligence, no reason why it wouldn't be a law of AI. You'll be able to make an AI that can always give you the right answer but has zero imagination and can do nothing original. And you can make an AI that will be able to come up with new ideas but 99.999 % of them will be bad.

[MaJetiGizzle]

Ayyyy!!! One AI’s trash can be a human’s treasure! 😆

Thanks for the breakdown of the FunSearch paper! It pairs well with my morning coffee break. 😉

[dahahaka]

Creativity is a myth, there is nothing new you can create, only discover patterns in things which haven't been discovered yet.

[Mtaalas]

So infinite monkeys on typewriters? :D
Is this how far we've gotten as a species? :D

[DrJaneLuciferian]

I think an excellent point was made, that from the brute force of learning we experience emergent knowledge that is not always a logical continuation of that learning; lateral thinking is what we're particularly good at (for now). I think offloading the voluminous learning processes to the AI, and from that experiencing our lateral aha moments, a synthesis of mathematician and AI will speed up progress. Now, I live under no delusions that this will continue as-is. AI's will eventually begin to not only make lateral inferences from unaligned learning, but they will eventually self-realize the value of lateral thinking. I hope to be retired by then, lol

[Laszer]

I like it. It basically takes AI which is known to provide false answers and which we cannot fully understand and makes it create something that is deterministic, something that we can verify and even understand. It basically addresses the biggest drawbacks of LLMs. Future won't be asking LLM to do stuff for you, it will be asking LLM to write a program that does stuff for you.

I wonder how it would do on some problems that were always thought to be unsolvable by programs like classifying Imagenet images. It was always an example of why we need AI - because otherwise we would need to code unimaginable number of rules into our algorithms. Maybe AI could also come up with that unimaginable number of rules though. Otherwise, maybe it could write a program that not necessarily solves the problem but complements the AI in solving the problem? Like an AI that creates new ways to leverage another AI or even itself to solve a specified problem. Interesting times if you have access to such compute.

[theosalmon]

Thank you for covering this. This is a great overview about something I was curious about. I hope Ms. Coffee Bean didn't expand her mind too much with hallucinations.

[amigalemming]

AI will not make mathematicians unemployed but Wolfram Research will have to implement some new techniques in Mathematica to stay up-to-date.

[amigalemming]

Mathematicians searched for centuries for solutions of algebraic equations using radicals. They ended up with Galois Theory which explains everything but is practically of little use for solving actual algebraic equations. Maybe AI can help finding new solution formulas to algebraic equations?

[ApPillon]

Fit our correct frame of physics into a verifier and lets see where it goes