Reward Hacking in AI

preview_player
Показать описание
Just like humans, artificially intelligent agents also strive to maximize their reward. Both humans and AI systems can get very good at gaming the system by finding loopholes.

0:00 Intro
0:19 Standardized Tests and Campbell's Law
1:04 Job Interviews
1:34 Academic Metrics
2:12 Reward Hacking in Artificial Intelligence
3:05 Reward Functions and Reward Shaping
3:58 Cobra Effect
4:32 Reward Tampering
5:17 Unforeseen Consequences
5:35 Outro

Related Articles:

Goodhart’s Law: Are Academic Metrics Being Gamed?

Faulty Reward Functions in the Wild

Learning Montezuma’s Revenge from a Single Demonstration
Рекомендации по теме
Комментарии
Автор

Real life examples (cobra effect, drug addiction, lobbying) are actually more representative than examples from field of ai research. Thanks for video

smaginandrew
Автор

awesome, I like these kinds of video snippets. Thank you for your contribution to this community. And keep doing the same :)

GauravSharma-uiyd
Автор

very informative video. Thanks.
Good analogies

rickmorty
Автор

Interesting take on things! The example of that game reminds me of a 2 Minute Papers video where an AI gamed a system to glitch through blocks. Good video too!

tonksonk
Автор

I'm a total ignorant when it comes to AI, but in the boat race example, wouldn't it be possible to fix the problem by requiring that the AI crosses the line within a certain time, or gets higher rewards to shorter it's race time is?

GrayCatbird
Автор

you're great. Thank's very much! Subscribed.

RagdollRocket
Автор

Great video! Do you have any videos on optimizers? I'm curious about your take on how optimizers can get stuck in wierd minimas like saddle points

saeidbagheri
Автор

Great video again. I dont doubt you can reach a lot more people if you keep this up. Just gotta get on the good side of the A L G O R I T H M once :). Maybe you could try to get rid of the echo, it would make the audio a lot more appealing. Looking forward to more videos!

abdullahkilinc