The Power of Intelligence - An Essay By Eliezer Yudkowsky

The Power of Intelligence is an essay published by Eliezer Yudkowsky in 2007.

Now, a few points:

Sorting Pebbles Into Correct Heaps was about the orthogonality thesis. A consequence of the orthogonality thesis is that powerful artificial intelligence will not necessarily share human values.

This video is about just how powerful and dangerous intelligence is. These two insights put together are a cause for concern.

If humanity doesn't solve the problem of aligning AIs to human values, there's a high chance we'll not survive the creation of artificial general intelligence. This issue is known as "The Alignment Problem". Some of you may be familiar with the paperclips scenario: an AGI created to maximize the number of paperclips uses up all the resources on Earth, and eventually outer space, to produce paperclips. Humanity dies early in this process. But, given the current state of research, even a simple goal such as “maximize paperclips” is already too difficult for us to program reliably into an AI. We simply don't know how to aim AIs reliably at goals. If tomorrow a paperclip company manages to program a superintelligence, that superintelligence likely won't maximize paperclips. We have no idea what it would do. It would be an alien mind pursuing alien goals. Knowing this, solving the alignment problem for human values in general, with all their complexity, appears like truly a daunting task. But we must rise to the challenge, or things could go very wrong for us.

▀▀▀▀▀▀▀▀▀PATREON, MEMBERSHIP, KO-FI▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

▀▀▀▀▀▀▀▀▀SOCIAL & DISCORD▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

▀▀▀▀▀▀▀▀▀OTHER STUFF▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

Bitcoin address: 1FX4iepZfh1yuMNYtvYf2CWL7gha8cakuf
Ethereum address: 0xDa8463494Dd233c3aBe59bc42Abc4D50823A5f3

▀▀▀▀▀▀▀▀▀PATRONS & MEMBERS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

Name
Colin Ricardo
Long Hoang
Tor Barstad
Stuart Alldritt
AGI275
Serena Thompson
Atm Lun
Ville Ikäläinen
Chris Painter
Fermain
Juan Benet
James
Dylan Mavrides
Michael McGuffin
DJ Peach Cobbler
Lex Song
Falcon Scientist
Trevaugn Martin
Please Insert Name
Christian Loomis
Ðœikhail Samin
AudD
Tomarty
Edward Yu
Chase
Ahmed Elsayyad
Chad M Jones
Jim Bob
Emmanuel Fredenrich
Neal Strobl
Renn Bright
bparro
Danealor
Dain Bramage
Travis Reid
Craig Falls
Diego
Aaron Camacho
Vincent Weisser
Shivanshu Purohit
Alex Hall
Hex Ramdass
Yarrow Bouchard
Ivan Bakhtin
Vincent Söderberg
joe39504589
Tim Davis
Oisin Hendrix
indexhtml
Craig Talbert
Klemen Slavic
Filip Passeri
Nick Sharp
hr101
Udo
Scott Alexander
Thomas Farago
noggieB
Dawson
Florian
Logic
Daniel Cunningham
YouAyePee
John S
Dang Griffith
Gabriel Ledung
William Deng
David Mc
Meade Marmalade
Ana Tomasovic
JHeb
Jeroen De Dauw
Craig Ludington
Jacob Van Buren
Gabriel Fair
Superslowmojoe
Lars Osborne
Nicholas Kees Dupuis
Ashten The Platypus
Michael Zimmermann
Austin Cluff
Nathan Fish
Sephiths
Ryouta Takehiko
Nathan
Kevin Ma
Bleys Goodson
FusionOak
Ducky
Treviisolion
Bryan Egan
Perry Jackson
Calvin McCarter
Matt Parlmer
Neel Nanda
Tim Duffy
Connor
Marty Betz
Robin Hanson
rictic
Mark Gongloff
Roborodger
ReddChronic
Matthew Brooks
marverati
Luke Freeman
Raphaël Lévy
Rochelle Harris
Dan Wahl
Francisco Lillo
AWyattLife
Evan Carroll
codeadict
Lazy Scholar
Torstein Haldorsen
Alex G
Supreme Reader
Michał Zieliński
The CEO
רם רינגל

▀▀▀▀▀▀▀CREDITS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

Animation team:
- ?? / :3

Writer:
- Eliezer Yudkowsky

Producer:
- :3

Narrator: Robert Miles
VO Editor: Tony Dipiazza

Sound design and music: Epic Mountain

[RationalAnimations]

The script used for this video is an essay published by Eliezer Yudkowsky in 2007.

Now, a few points:

Sorting Pebbles Into Correct Heaps was about the orthogonality thesis. A consequence of the orthogonality thesis is that powerful artificial intelligence will not necessarily share human values.

This new video is about just how powerful and dangerous intelligence is. These two insights put together are cause for concern.

If humanity doesn't solve the problem of aligning AIs to human values, there's a high chance we'll not survive the creation of artificial general intelligence. This issue is known as "The Alignment Problem". Some of you may be familiar with the paperclips scenario: an AGI created to maximize the number of paperclips uses up all the resources on Earth, and eventually outer space, to produce paperclips. Humanity dies early in this process. But, given the current state of research, even a simple goal such as “maximize paperclips” is already too difficult for us to program reliably into an AI. We simply don't know how to aim AIs reliably at goals. If tomorrow a paperclip company manages to program a superintelligence, that superintelligence likely won't maximize paperclips. We have no idea what it would do. It would be an alien mind pursuing alien goals. Knowing this, solving the alignment problem for human values in general, with all their complexity, appears like truly a daunting task. But we must rise to the challenge, or things could go very wrong for us.


You can support Rational Animations on:

[DineshGaikwad]

I really liked the line, "A blank map does not mean an empty territory." Amazing insight.

[nisenobody]

I can't believe that charisma doesn't come out of the kidneys 😢

[smitchered]

I can't properly describe how much I like essays like this. The analogies coupled with beautiful hilarious and clear graphics makes everything click into place. The video also strikes fear in me, but it's productive fear, as in thanks to you I'm now going to spend the rest of my afternoon figuring out what's up with AGI.

[AnythingMachine]

To be faîr, the squishy things did have pretty good endurance running and hand-eye coordination they were doing fairly well for themselves, even before they attained ultimate power

[NaviaryMusic]

"Within that gray wet lump is the power to search paths through the great web of causality, and find a road to the seemingly impossible."

Single most incredible quote in the whole essay. I shed a tear. To think humans have this capability- To dream, and to find a way to fulfill said dream even against all odds of the universe, is truly incredible. In a constant battle against entropy, where everything wants to be at equilibrium, we find intelligence fighting to put order to the universe.

[eternisedDragon]

"They had no armor. They had no claws. They had no venoms."

I had to laugh when I heard this, because it sounded like a portion about Lanky Kong straight from of the DK rap:
"He has no style... he has no grace. This kong... has a funny face."

[vev]

Wow, this video was incredibly thought-provoking. It's fascinating to think about the power of intelligence and how it has allowed human beings to achieve such incredible things. I also found it interesting to consider the framing problem around commercializing AI, and how our current understanding of intelligence may be limiting our ability to imagine its full potential. Overall, a great essay by Eliezer Yudkowsky.

[thegwangster]

I love this channel. I hope helps you make some more.

[maickelvieira]

i am always amazed to how profound those videos are, every video is a piece of art on itself, great job and thanks

[kikivoorburg]

It can be difficult to remember how amazing our very existence is, it’s good to have reminders like this! (And indeed to consider the impacts a true AI might have.) Keep up the great work!

[bennythetiger]

I am a software engineer. I told my mom I chose this career because I liked the idea of creating things that don't exist yet, bringing uninvented concepts into reality right at the technological frontier. It is extremely fascinating how powerfull and complex intelligence is. We built cities, cultures, entire societies and ever-increasingly sharper technology from the ground. It always amazes me to think about human civilization and their unfathomably intricate systems. Railroads, transportation networks, commerce, currencies, stocks, supply chains, politics, social networks, error-handling mechanisms, space exploration, energy production, nuclear power, computers, software, chips... Everything and everyone working at the same time, interconnected, and dependent on one another. Of course, we have had numerous instances where we made the greatest mistakes of our existence, but that's exactly why we should search even more for knowledge and embrace the responsibility

[austintheultimate]

I love that point. Just because we haven't learned something yet does not mean it doesn't exist. I think too many people forget that.

[MrMyers]

This video brought on the realisation that if any true intelligence is born from AI, then it will be in the hands of businessmen. That realisation is even more reason to be opposed to AI development. It’s like allowing companies to develop their own nuclear weapons programs, nothing could possibly go wrong and I’m sure these companies have the interest of the whole of humanity in mind!

[insightsforimpact]

Excellent two-part video series! An astonishing amount of effort went into these, and it has paid off!

[Hansulf]

I always thought that our most incredible power is to make things that seem impossible into possible, from dream to reality, with the power of intelligence and creativity. Beautiful video!

[lebraza]

This felt so satisfying to watch, great job making this video!

[prolamer]

Can't express fully in this yt comment format, but your video is accurate most people including ones building these systems fail to imagine results of intelligence only order of magnitude stronger than ours and its power go for sure much higher where even I cannot see. After much deliberation I concluded this... we cannot give such mind any self awarness or self preservation directive because in that very instant endless blessing becomes nighmare. Not because it is "evil" but because it is evil and near impossible to turn such thing off. And once you can't turn something "off" well even simplest of things like fire becomes deadly fast let alone always improving mind.

[nhpkm]

Great video .

My testable definition for intelligence is ability to 'predict the future '/ 'limit uncertainty ' .

If I eat healthy food I'll live longer is an example of a very smart future prediction .

[julian]

I love how in this very serious topic is explained using words like squishy things