Season 1 Ep. 22 OpenAI's Ilya Sutskever: The man who made AI work

On the last episode of Season One, our guest is Ilya Sutskever. Ilya is the Co-Founder and Chief Scientist of OpenAI. As a PhD student at Toronto, Ilya was one of the authors on the 2012 AlexNet paper that completely changed the field of AI, resulting in the widespread adoption of deep learning and the avalanche of AI breakthroughs we’ve seen the past 10 years.

After the AlexNet breakthrough in computer vision, at Google, among many other breakthroughs, Ilya showed that neural networks are unexpectedly great at machine translation, at least at the time it was unexpected, now it’s long become the norm to use neural nets for machine translation. Late 2015 Ilya left Google to co-found OpenAI, where he is Chief Scientist. Some of his breakthroughs include GPT, CLIP, DallE, Codex. Ilya’s academic work, less than 10 years out of his PhD, has ben cited over 250,000 times, reflecting his absolutely mind-blowing influence on the field.

What's in this episode:

00:00:00 Introductions
00:03:00 Why take a closer look at neural net works originally?
00:08:25 What was going through Ilya's mind during the AlexNet discovery?
00:18:25 Ilya's early years
00:21:19 How Ilya stayed motivated
00:29:07 Sam Altman and the beginning of OpenAI
00:36:22 LSTM models and reinforcement learning
00:56:06 How will our productivity change?
01:00:22 Instruction-following models
01:12:13 Ilya's vision of the future of work
01:16:14 Ilya's advice to be productive

| SUBSCRIBE TO THE ROBOT BRAINS PODCAST TODAY |

Host: Pieter Abbeel
Executive Producers: Ricardo Reyes & Henry Tobias Jones
Associate Producer: Alice Patel
Audio: Kieron Matthew Banerji
Video: Bo Obradovic
Music: Alejandro Del Pozo

[RickardHallerback]

He is so pedagogical in the way he speaks. He makes it easy to follow and understand. Only someone who knows what they are talking about can make these types of clarifications on these different complicated topics.

[AlecsStan]

These people are historical figures. These are the giants that give their shoulders for future generations to stand on.

[ELECTRHERMIT]

This is what TRUE GENIUS looks and sounds like folks. Not sitting around theorizing, but actually solving and making a next level invention. I'm so glad these guys are so young, we need a lifetime of young innovators perfecting AI, which will in turn perfect AI which in turn will perfect AI and create a self-realization system that improves exponentially. We will need this to save the planet and develop medicine and power supplies and other future challenges. I'm very very grateful and excited because things are taking off right this very minute. Hang on tight.

[nichengmo]

He is literally the smartest person behind Open AI.

[fanyang]

THE best interview about LLM, GPT, OpenAI and Illya's great work!
Thanks a million!

[AADVS-RPY]

Listening to Andrew Ross Sorkin discuss the early days of OpenAI with Greg Brockman is eye-opening. It's fascinating how they saw a gap in AI development and seized the opportunity to create something innovative. The way they tackled complex challenges like training bots for Dota, against all odds, shows the power of their approach. It's inspiring to hear how they pushed the boundaries of reinforcement learning, debunking initial skepticism with each milestone. Definitely a must-watch for anyone curious about the intersection of AI and gaming!

[JousefLITE]

Definitely one of my favorite episodes in this season! Thanks Pieter!

[OsvaldoQuino]

Hello there... Ilya, good to see you again, Mr Pieter, Thanks for you program, looking forward to see it more you program. 👍💯👍🥃

[balapillai]

Ilya Sutskever. He who inflected AI. The magnitude genius.

What great inspiration and company to my toil on steps to infinity & the source of pessimism.

The first village in human evolution would not be possible with the amount of pessimism we have now. Let this sink in!

[briancase]

It's interesting to listen to this today just for what it is, of course, but also because ChatGPT was, at least from the perspective of the wider world, a year away. Thanks for the interview.

[rrrbb]

Jeff Hawkins was going on about prediction being the key to everything; in an old video from maybe a few years ago. That always stuck with me. If you have senses and actuators, they are equivalent to current input, and current output. If your brain predicts how something will feel before it is touched, or predict what you will see in the next frame; then you have a generalized basis for your senses to supervise the learning. It is definitely intuitive when it's a matter of predicting the next word.

[JoePiotti]

So when AI makes a Time Machine, this is the guy we will have to go back in time to stop.

[TheYashakami]

This is one of the better interviews I've seen. Well done

[prabhavkaula]

Thank you so much Pieter sir. This podcast is a gem.

[emiliomorales]

Thank you for this episode. Also It would be great listen to Alex Graves and Aäron van den Oord

[osalicsomonic]

Absolutely great talk! Thank you. Must Rewatch, I guess.

[raghavendrakaushik]

At 22:14, what is the name of book and its author?

[oneflow]

Hi, Pieter, your podcast is very inspiring to me. I wonder whether we can translate the closed captions of some podcast into Chinese and publish the content as blogs. We will keep the original link of video in the blog. I believe many others like me will find the content inspiring but didn't know your podcast before.

[carvalhoribeiro]

Great conversation. Thanks for sharing this

[LoisSharbel]

Thank you for this fascinating interview. It is dense information for me, with NO study in these fields at all, yet with your charming enthusiasm, interesting questions and Ilya's meticulous answers, I am able to glimpse the processes that have led to such exciting discoveries. I am in awe of individuals like this man, you, Geoff Hinton and Elon Musk, who are openiing our minds to horizons unimagined even recently. THANK YOU!