General AI Won't Want You To Fix its Code - Computerphile

Part 1 of a Series on AI Safety Research with Rob Miles. Rob heads away from his 'Killer Stamp Collector' example to find a more concrete example of the problem.

This video was filmed and edited by Sean Riley.

[TS]

"Nobody's truly happy, you'll never be happy or accomplish your goals."

Wow, computerphile cutting deep

[flensborg]

"But the only important thing is stamps" literally made me laugh out loud. This was a great video.

[miketothendpwr]

"being given a new utility function will rate very low on your current utility function"

That got me laughing hard.

[ioncasu]

"You can't archieve something if you're destroyed".
Much deep. wow.

[darkmatertm]

"The most important thing in the world is stamps. If you change me not to care about stamps, I won't collect enough stamps"

I understand my fellow humans so much better after hearing this. Having a closed mind is very rational, as it turns out.

[algee]

after watching a bunch of these, i must say rob is so satisfying to listen to. i can literally feel the knowledge being tetris fed into my brain.

[dbsirius]

1:55 Evil laughter, then hammering sounds....

[wheeleybobs]

"Stop what you're doing and do something else" ... NO .... "sudo stop what you're doing and do something else" ... OK

[GetawayFilms]

Someone got excited and started making things in the background...

[Zerepzerreitug]

I'm always happy when there's a new Computerphile video with Rob Miles :D I love general purpose AI talk. Especially when it makes me shiver XD

[jameslay]

Man, these videos are so important now. 6 years ago, I thought something like chatGPT was decades away.

[elkanaajowi]

This is very interesting and now, 6 years later, we are beginning to have the AI Black Box Problem.
An episode on this will be highly appreciated. It is terrifying to know, say, your self-driving car works in a way you and its designer cannot understand.

[cubicinfinity]

I used to think that the AIs depicted in science fiction were way off base from reality, but the more I've learned about AI the closer I realize sci-fi has always been. I don't know how far it was correct, but it is much closer.

[razorbelle]

A more succinct way of describing the problem to laypeople would be to say that an AGI would never deliberately enter a position in which you are both able and willing to change or obstruct its goals. If it would do otherwise, then it isn't a sufficiently general intelligence (viz., it neglects human psychology).

It would know to deceive you into believing that its existing goals and consequent behaviors are compatible with what you desired it to do, but execute the goals as explicitly specified and understood—even though it understands that it is not what you wanted—as soon as it has circumvented the threat of destruction, which is its primary instrumental goal.

[centurybug]

I love how the background is just filled with sounds of destroying things and laughter :P

[warhammer]

i could listen to Rob Miles' conversations about AI for hours.

[msurai]

Extremely interesting topic. I'm looking forward to the rest of the series.

[valeriogalieni]

Robert's videos about AGIs are fascinating, enlightening, and scary af

[hakonmarcus]

Can you not give the machine a utility function that makes it want to obey an order? I.e. when it is collecting stamps, it is only doing this because the standing order is to collect stamps, whereas when you tell it to stop so you can look at its code, this order replaces the previous one and it will only want to stop and let you look at its code?

You could probably also build in failsafes so that the AGI only wants to follow any order for a short amount of time, before it chucks the order out and needs a new input. I.e. collect as many stamps as possible until next friday, then at this point await a new order?

[isuckatsoldering]

The editing here is brilliant in the way that you return to his "original" statement once you're equipped to understand it.