2 Years of My Research Explained in 13 Minutes

Outline
0:00 - Intro
0:34 - DreamerV3
1:23 - State representation
3:27 - Research problem
4:26 - World model experiments
9:21 - Policy learning experiments
13:22 - The paper

Social Media

Sources:

[JoeTaber]

My Policy model appreciates good jokes about subscribing.

[dhruvxx]

doing literature survey or just getting into a field would 1000x easier if somehow in an ideal world we could have videos like these, and thank you so much, your effort which you put into making this vid while being in academia too!!!

[IxCIHAoX]

Really cool, congratulations. You can really be proud of yourself.

I did just the course work and thesis, as what is what we are supposed to in my uni. And now after graduating i really regret never doing anything more in that regard. Videos like yours really motivate me to try researching in my freetime. But I still don't know where to start really, and fear it will be a waste of time. But videos like yours, people at the start of their academic career, give me hope. Thanks

[firewind]

You have a real knack for presenting complicated things in an easy to understand/digest manner man! Your research is awesome and I can’t wait till I can pick up a ejmejm brand robot third arm

[andrestorres]

I love the way you approach research. The questions you ask are very good!

[pieriskalligeros]

Edan you're just amazing dude! Nobody asked you to explain AE compression, bottlenecks, latent states, and yet you did! Love your format and your research, keep shinin' and keep educating on YouTube with your iPad! RL Professors keep note (Looking at you slide ppt addicts!) Also, as you were explaining the 1 step (TD0?) action prediction, I couldn't help but wonder if LLMs (say GPT-2) could just be model-based RL agents!

[ai_outline]

Great video! I wish there were more computer science channels like this!! This being free is amazing. Thank you ❤️

[gobblegobble]

Dude, your 13 minutes of video literally explained more than a whole semester at uni!!! Thanks!!!

[ramzykaram]

You're the first channel to ask the questions i have in my mind which always push me to read more papers, why things work !! Usually channels and books focus on the tutorial and catchy titles. I have set notifications of your channel to all and hope to see more videos

[rmt]

This amazing, and puts an explanation to a concept I've been pondering on. Would be a great start to a self-simulating module for an AI, even if it only works on a projection like a video game state. You gained a new subscriber!

[thebarrelboy]

Really nice to see a new video from you Edan, keep it up! You're finding your style with the videos, both artstyle and script-wise, and I love it, I think this is one of your best videos so far :)

This topic is super interesting as well, I based my master thesis on the precursor to DreamerV3, PlaNet, and its so cool to see these deepdives into the model-based RL methods, keep it up!

Please don't dumb down your videos more, you've found the perfect sweetspot now in my opinion, where we can actually learn the small intersting advancements of different methods and papers, without going down into the maths and formula derivations, its perfect!

[TheSMKD]

Wow this is great research progress. Thanks for sharing here on youtube!

[CC.unposted]

It’s similar to using maximum and minimum functions in mathematics for various tasks ( if your counter and button that increases the counter are offset, you can use the maximum value of the counter to get the result). Instead of an offset, you might use clamping effects, where values are restricted to 0 or 1 rather than more precise values. Given that the environment may introduce noise, especially for modal values, it could be easier to obtain values, though it might be coincidental. Additionally, editing fewer values is typically easier than editing many. While continuous adjustments can lead to better learning over time, it takes longer due to the complexity of optimizing many digits.


here by noise I mean Pesudo Noise possibly

[AnalKumar]

This is an excellent video showcasing your research. I wish more people make such videos of their papers (I know I am going to once my paper is published).

[dylancope]

Very interesting work! Congrats on being accepted to RLC 😊 and thank you for sharing.

My research is in MARL and communication between agents. I generally work with discrete communication symbols for various reasons, and I've often argued for intuitive benefits of discretisation (even in noise-free channels). It's nice to see some concrete work that lines up with my intuitions!

[joshuasonnen]

THANK YOU! You explained model-based RL than anyone else! I've been trying to learn this for a month

[keshamix_]

Simply wow! An amazing video. I’m not really in the field, not even a uni student yet, just curious about technology, but I feel like I understood most of what was mentioned there! And it only proves the point that you are an amazing narrator, good job!

[thatotherguy]

I wish I understood more than 1% of what you said, but from what I did get, it looks very impressive. Congratulations.

[AIShipped]

Please make more of this type of video on this type of subject. One of the first times I was truly interested in a video

[ryanstout]

Great work! I remember looking for papers a few years ago to explain why discrete representations outperformed continuous in many situations. There wasn't much research on it at the time. (I did a few experiments and came to the same conclusion that it enabled a model with less parameters to learn more efficiently, but if you weren't parameter constrained they would both get to the same point) Looking forward to reading the paper!