AI exploits a glitch in Trackmania

I trained an AI in Trackmania with reinforcement learning, and tried to make it learn the hardest technique in this game: the noseboost.

To support my work on Patreon:

Note: This video was re-uploaded a few hours after the original upluoad due to a copyright issue with the music

Contact
• Discord: yosh_tm
• Twitter:  yoshtm1  

Link to download the last map:

TAS runs shown in the video:

Other reinforcement learning projects mentioned in the video:

Thanks to Donadigo, for TMInterface!

The setup used in all maps to put the car on its nose was inspired by this map:

On a more general note, this video would not have been possible without the passion and creativity of the TAS community, who discovered the noseboost in 2021. Special thanks to them :)

You can find a list of the musics I used at the end of the video.

[yoshtm]

Thanks for watching ;)

To support this work: patreon.com/yoshtm

[jakubpluhar]

Honestly the coolest thing about this video is how well it showcases the issues with reinforcement learning. The AI is the best ever at getting rewards. It will do anything and everything in order to get the most reward and that includes doing exactly the opposite of what you actually want. You constantly need to change and improve your reward systems in order to actually get the AI to do what you want it to do. It's like a really smart toddler

[Coconut-]

* The year is 2045, and you're waiting for your AI self-driving taxi to arrive *

your self-driving taxi:

[KonnorGann]

This is the highest quality editing and storytelling I’ve ever seen. Beautiful music, voiceover, and all around impressive production value. Well done sir.

[firenyth]

I loved seeing the AI dancing around the carrot then flowing over to the new carrot target, that was beautiful.

[srandom]

Imagine driving around that open field map with the target position constantly set to your car.
The eye of a hurricane of trackmania racers.

[lifeisatrip_lavitaeunviaggio]

You're a genius. This video is very entertaining and well done. Thank you! Grazie from Italy

[haiden]

16:55 seeing a swarm of car spinning around a pole at 1000 kmh like a summoning circle or ritual give me shivers

[Teckstudio]

Awesome work! Here's a little carrot for you. 🙂

[Keyboardsheep]

Amazing production quality and really cool content! Nice work!

[caebyt]

"Hey I've seen this one before"
"What do you mean you've seen it, it's brand new!"

[JohnDeWill]

Not only is the AI stuff absolutely incomprehensibly insane, but the editing is also above next level, massive props man

[FluffyDaWuffy]

The dedication to this is unreal. I want to go and watch this AI from when you first started to experiment with it. I want to see more amazing feats accomplished, like if the AI can do Deep Dip 2 and make it really fast 😊 a wonderful sight this is.I couldn't support on patreon as of this time so take it here instead.

[Henry]

The editing on this is insane, thank you for putting in so much work!

[grexursorum]

This is amazing! Thank you so much. Next target DeepDip... 😀 Jokes aside... I love how much effort you put in this. Thats pure academia. Doing research on a topic where the use is basicly unkown, just to see what will happen and what can be learned. Thats the kind of mindset, which shaped our world. Keep on searching. I will follow to see what comes next.

[MCCompanyPL]

11:20 I love the wirtual reference, 10/10

[multi_array]

thats why there was a deleted video in the watch later list. i wish its possible to tell what the deleted videos are.

[rakly]

1:18 Would have been so funny if he said "I spent almost a whole year aswering the question, and the answer is 'yes, yes it can'". - And then the video ends :D

[khusanakramkhodjaev]

I think you are closely getting into understanding the game design behind the Trackmania and some core principles of physics - gyroscopic movement; a case of conservation of angular momentum interacting with unintended game physics. If the front wheels are spinning rapidly while the rear wheels are off the ground, the system might behave like a gyroscope or a spinning top, where the rotational inertia stabilizes motion and resists external forces. One hypothesis is that the game engine miscalculates friction and collision forces at high angular velocities, allowing continuous acceleration. Another possibility is that it creates an artificial energy feedback loop, sustaining rotation like a perpetual top. If real-time physics data—angular velocity, inertia tensors, and contact forces—can be extracted, it might reveal whether this is true gyroscopic stabilization or a numerical instability in the physics simulation.

[DJFelixChester]

In reference to the AI preferring the center of the field during testing even if it can’t see the walls, think about monarch butterflies. They have a migration route that takes multiple generations to complete, and even involves directing around lakes which no longer exist. While that particular generation doesn’t see the wall, part of its reinforcement might be directing it surreptitiously to stay as central as possible.