Build an Mario AI Model with Python | Gaming Reinforcement Learning

Teach AI to play Super Mario

In this video you'll learn how to:
Setup a Mario Environment
Preprocess Mario for Applied Reinforcement Learning
Build a Reinforcement Learning model to play Mario
Take a look at the final results

Links:

0:00 - Start
0:18 - Introduction
0:44 - Explainer
1:58 - Client Interview 1
2:02 - Animation 1
2:30 - Tutorial Start
3:22 - Setting Up Mario
10:44 - Running the Game
18:26 - Understanding the Mario State and Reward
20:44 - Client Interview 2
21:38 - Preprocessing the Environment
26:22 - Installing the RL Libraries
31:11 - Applying Grayscaling
35:32 - Applying Vectorization
36:56 - Applying Frame Stacking
42:46 - Client Conversation 3
43:05 - Animation 3
44:00 - Importing the PPO Algorithm
47:33 - Setting Up the Training Callback
50:13 - Creating a Mario PPO Model
55:30 - Training the Reinforcement Learning Model
1:02:40 - Client Conversation 4
1:02:56 - Animation 4
1:04:01 - Loading the PPO Model
1:06:10 - Using the AI Model
1:15:56 - Client Conversation 5
1:16:37 - Ending

Oh, and don't forget to connect with me!

Happy coding!
Nick

P.s. Let me know how you go and drop a comment if you need a hand!
#ai #python

[amanlohia]

Your videos are freaking amazing. I m currently a senior undergrad and will start my career in applied ML next year. Love watching your videos and learning.

[doremekarma]

for people who are getting following error at 15:14:
not enough values to unpack (expected 5, got 4)
you can fix it by changing the line at 10:44:
env =
to
env = gym_super_mario_bros.make('SuperMarioBros-v0', apply_api_compatibility=True, render_mode="human" )

[offstyler]

Maybe you already know this by now, but by wrapping your env in the Monitor Wrapper by SB3 you can get additional info in your Tensorboard log like how much reward your model is collecting and how long the episodes run.

[windrago]

much of this tutorial no longer works (due to updates on ALE and gym) it was well done but for the purposing of learning it would be great to have a refreshed version - great work nonetheless

[itsbenteller]

Crazy how in just two years everything breaks

[vakurbarut]

Newer version nes_py and gym cause some bugs as 1/1/2024 here is how I managed to run the code:
1.
JoypadSpace.reset = lambda self, **kwargs: self.env.reset(**kwargs)
env = gym_super_mario_bros.make('SuperMarioBros-v0', apply_api_compatibility=True, render_mode='human')
env = JoypadSpace(env, SIMPLE_MOVEMENT)
2.
done = True
for step in
if done:
env.reset()
obs, reward, terminated, truncated, info =
done = terminated or truncated
env.render()
env.close()

Hope it helps!

[ciaranmalone]

Just to note you will need to set !pip install or you will get a render_mode error

[Seriosso]

Was waiting for such a video for so long! Your videos shouldb be taught in CS classes! Thank you for the great effort!

[AdvancedNoob]

Couple of things that were not working for me, hopefully it will help others, It's 2023 so some of the API libraries etc are behaving a bit differently, but it's a nice tutorial nonetheless.

My changes are as follows:

[DK-]

after all preprocessing steps when i try to run env.reset() it pops an error -> TypeError: reset() got an unexpected keyword argument 'seed'
can anyone help me to rectify it as i cant proceed without rectifying it

[lamtungphung-cqbn]

Can someone help me with my bug? Why does my state return a tuple? It shows exactly what it suppose to return in the video but inside the (). This prevents me from checking the shape since we cannot check the shape of a tuple. Thanks for helping me!

[_obegi_]

For those that are getting an error saying expected 5 got 4. Change the line where you are taking the step to this.

state, reward, done, info, _ =

Adding the underscore makes it so all other variables are ignored, making the expected more error go away

[darkwave_]

Great video again!! I learned already a lot from your videos. Big thanks for all the effort !! For future vidoes I would like to see how to advance to a next level with RL. I read something about concept networks where an RL agent can be trained to do a bunch of dedicated tasks. Here in the mario game that would be i.e. jump on the enemies, jump over hole, jump on pipe, collect coins, .... It would be interessting to see how to teach an RL agent some kind of skills instead of just letting the algorithm do all at once. Something like a 2 layer approach where first the needed skills get trained and then secondly by using the skills the complete tasks gets trained.

[johanneszwilling]

🤯 Superbly done!! Really appreciate the commenting and explanations. You know your stuff!

[geriain]

I really hope that you will never stop making such great videos! Cheers up man and happy Christmas!

[ndiritu_nyakiago]

Where did you get the resources to teach this tutorial from Nick? As of august 2023 most of the parts in the video does not work anymore because of version differences and lots of updates but the content is great.I will very glad when you help me

[avivbenari]

on 37:10 when trying to implement env #4 i get:
AttributeError: 'SuperMarioBrosEnv' object has no attribute 'render_mode' .
can anybody think of a solution?

[Ohnomcrose]

I got the game up and running with PYthon with env = gym_super_mario_bros.make('SuperMarioBros-v0', apply_api_compatibility=True, render_mode="human") COMBINED with state, reward, done, info, _ = otherwise it would not work. Thought I got through the hard part but then I could not run env.reset() after stacking the frames in the preprocessing part.

[adityapunetha]

We can also simplify the input by maybe removing the background image, flattening the pixels into just black or white instead of the 8 bit range and maybe replacing the blocks and enemies with just white squares and black squares(this would require a new gym environment), this simpler input might be easier for the agent to understand and might learn faster

[Yupppi]

My favourite part of this was "NameError" from gym_super_mario_bros and SIMPLE_MOVEMENT and env even though I installed both gymsmb and nespy before using jupyter-lab but still trying to include the pip install like in the tutorial. A classic situation of followign a coding tutorial and copying every move, but getting errors all over. Until I learned I have to click play on every cell from the beginning in order. Just never managed to get that screen of python playing mario. (Until I read comments and tried a couple of fixes).

I bet this would be a pretty nice tutorial if every cell wasn't full of red and the left side brackets weren't full of * and me having no idea if it does anything at all.