Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition

This conversation between Geoffrey Hinton and Joel Hellermark was recorded in April 2024 at the Royal Institute of Great Britain in London. An edited version was premiered at Sana AI Summit on May 15 2024 in Stockholm, Sweden.

Geoffrey Hinton has been called “the godfather of AI” and is considered one of the most prominent thought leaders on the emergence of artificial intelligence. He has served as a faculty member at Carnegie Mellon and a fellow of the Canadian Institute for Advanced Research. He is now Emeritus Professor at the University of Toronto. In 2023, Geoffrey left his position at Google so that he could speak freely about AI’s impact on humankind.

Joel Hellermark is the founder and CEO of Sana. An enterprising child, Joel taught himself to code in C at age 13 and founded his first company, a video recommendation technology, at 16. In 2021, Joel topped the Forbes 30 Under 30. This year, Sana was recognized on the Forbes AI 50 as one of the startups developing the most promising business use cases of artificial intelligence.

Timestamps
Early inspirations (00:00:00)
Meeting Ilya Sutskever (00:05:05)
Ilya’s intuition (00:06:12)
Understanding of LLMs (00:09:00)
Scaling neural networks (00:15:15)
What is language? (00:18:30)
The GPU revolution (00:21:35)
Human Brain Insights (00:25:05)
Feelings & analogies (00:29:05
Problem selection (00:32:58)
Gradient processing (00:35:21)
Ethical implications (00:36:52)
Selecting talent (00:40:15)
Developing intuition (00:41:49)
The road to AGI (00:43:50)
Proudest moment (00:45:00)

Follow Sana

[jamesperez]

The most pleasant and articulate voice in the entire ai space right now. Absolutely no jargon, just clean and crisp explanations, only using terms for big concepts when necessary.

[ItCanAlwaysGetWorse]

Listening to Jeffrey Hinton is such a joy. He seems to me one of the most authentic and transparent souls among the famous people. There are many talented beings worth of admiration. But If I were given the choice to meet and share time to talk with some of them, he would be absolute first in my list.

[keyvet]

Well deserved Nobel Prize, Professor. Proud to be living in the same country as you !

[jianjielu]

one of the best interviews I have ever watched!

[dimzen]

Ability to explain complicated things in a simple way is sign of deep understanding. Best interview I know so far in this field.
And offcause right questions supported it.

[mreza]

Wow. All the questions were just great. Thanks for asking them.

[ReflectionOcean]

By YouSum Live

00:01:16 Early disappointments in brain understanding.
00:01:59 Influence of Donald Hebb and John Fornoyman.
00:02:33 Brain learning through neural net connections.
00:04:13 Collaborations with Terry Sinowski and Peter Brown.
00:05:08 Encounter with a young, intuitive student, Ilia.
00:06:00 Ilia's unique perspective on gradient optimization.
00:08:02 Scale and computation's impact on AI progress.
00:08:25 Breakthrough in character-level prediction models.
00:09:01 Neural net language models' training insights.
00:10:36 Integration of reasoning and intuition in models.
00:12:46 Potential for models to surpass human knowledge.
00:17:16 Multimodal learning enhancing spatial understanding.
00:18:18 Impact of multimodality on model reasoning abilities.
00:18:40 Evolutionary perspective on language and brain synergy.
00:18:41 Evolution of language and cognition.
00:18:57 Three views of language and cognition.
00:20:12 Transition from symbolic to vector-based cognition.
00:21:36 Impact of GPUs on neural net training.
00:23:13 Exploration of analog computation in hardware.
00:25:31 Importance of diverse time scales in learning.
00:27:37 Validation of neural networks' learning capabilities.
00:29:12 Inquiry into simulating human consciousness.
00:36:01 Brain's potential use of backpropagation for learning.
00:37:42 Brain's learning potential and beneficial failures.
00:38:02 AI advancements in healthcare for societal benefit.
00:39:00 Concerns about misuse of AI by malevolent actors.
00:39:23 International AI competition driving rapid progress.
00:40:03 AI assistants enhancing research efficiency and problem-solving.
00:41:51 Intuition in talent selection and diverse student profiles.
00:42:00 Developing intuition by filtering information effectively.
00:43:26 Focus on big models and multimodal data for AI progress.
00:44:08 Exploration of various learning algorithms for AI advancement.
00:45:11 Pride in developing the learning algorithm for Boltzmann machines.

By YouSum Live

[bleizthomas]

At various points in this discussion you can see the interviewer smirking. He is reminded again and again how insightful and articulate the man in front of him is. Great video.

[xuanchili]

"these big neural nets can actually do much better than their training data." things like this mentioned in this talk challenge us to look over the concepts we previously missed. This is by far one of the best interview from Hinton.

[GeorgeMonsour]

Stunned by the prediction of creativity! Yet it makes sense because creativity is separate from intelligence or self awareness. Hinton is a reminder of the great geniuses of the past s/a Faraday, Gallileo, Da Vinci, ... I think he has to be considered for that continuity.

[maxwang]

Geoffrey is so pleasant and such a joy to listen to. Love him.

[kimyunmi]

Nice to see the interviewer was also standing for the whole interview in honour of hinton's back pain. Always remember PSR (principle of sufficient reason): there is reason behind every event.

[executivelifehacks]

Hinton is great. If you, with 4k subs, can get Hinton, please try to get Sutskever.

[peterwang]

Purely by speaking, he transfers much more insight and information than I would see in most papers

[ShaifBasier]

Good job of the interviewer, kudos for Mr. Hinten surely. Nobelprizes pale … honestly.

[dhamovjan]

For the first time in my life, i see an interview and the sensation of "Fantastic questions" keeps popping up.
Thank You!

[capitanalegria]

A briliant, brilliant man with enough imagination tu fill up the universe, perhaps even decipher the essence of thoughts, yet the humility of someone who still holds ingenuity and admiration in every observation. A treat to listen to. ty

[davidderidder]

Geoffrey strikes me as a genuine ethical human, I hope he never hesitates to be open about observed dilemmas.

[SkysMomma]

Fantastic interview! The questions were awesome, the answers profound.

[amritbro]

The work of Geoffrey Hinton on backpropagation is remarkable and has significantly accelerated the progress of AI, as we are experiencing today.