Teaching an AI to Beat You At Your Own Game!

I'm trying to teach an AI to play a simple game and beat me at it.
For that I'm using Unity 3D and machine learning.

More specifically: An evolutionary algorithm that evolves the AI over time.

The training took about 50 hours. Unfortunately it didn't quite reach super human levels. Not quite sure what went wrong...

In this video I want to tell you what I found out about machine learning so far and show you my results.

[AdroitConceptions]

from what I do know... you need to train sets of AIs with perumutations, not just a single one with this method... else it is only training against itself.
going only 1 generation (or even a few) back, isn't likely to be making a large variation each time, so it is still 'learning to pay against itself'
from waht I remember about genetic alogrthyms like this, you need sets, and then you take the best from each set + a few random ones from that set and then run some mutations on each to create the next generation.

[JonasTyroller]

Hope this was interesting. Let me know if you have any questions. :)

[Xonatron]

Love how you show the AI visually in the game!

[jakejollimore]

This was actually a great explanation of a neutral networks. Thanks!

[CalebWillden]

Solution to your processing power problem: Just convince Squid to help and simulate millions of training environment universes. 😄

Loved the explanation of neural networks. I think I kind of get it somewhat now!

[Stehocke]

Very interesting video, I really enjoyed watching it - smart visualisation approach. You used a very straight forward solution as evolutinary algorithm. A reason your trained AI failed is maybe that your best version is trapped within a local optimum. Slight mutations are reasonable in the sense of a directed search, however, it not capable of escaping from pronounced local optima. As mentioned in comments before inceasing your population size could also improve the result. You may utilize a round-robin to mimic the traditional fitness value of a chromosome. Subseqently, you keep exclusively the best chromosomes/ais into the next generation. As mentioned before, due to the small population and slight mutations, you train to play the ai against itself or early versions, respectively. Consequently, you obtain a one-trick-pony which cannot adapt because it is only used to play a mirror.

[Keviamaya]

Who would win, a 50 hours trained Machine Learning AI or this one line boi!?

[sapmanp]

I have seen so many videos on AI and never understood how it worked but you explained it extremely well. It finally doesn't seem like black magic anymore

[rockybalboa]

it's over jonas, he has the high ground

[Pepe]

I think this type of stuff is super interesting. :D I knew about the basic process before, but I have not seen a real experiment with it. Keep going! :D

[programerkelly]

Fascinating!
The only problem is, that this wouldn't work for a more complicated AI.
It would be great to see two AI soldiers training them selves in my simulator,
but they would have to do so many things, that it would take too long and
a manually written AI could do the same things better.

But it's a good idea to test the AI. For example, I think I will setup a training
map where you have will have a shooting range and a room to room close
quarters training area, that you could use to train your self. And it would
give you feedback on your accuracy, speed and time spent being visible
(not behind cover). This would be a good training tool and also could be
really useful to see how effective the AI is.

First I could run a simple AI on the shooting range.
Can it seek cover and hit the targets?
Than I could implement a room clearing script.
It would simply follow one of the walls and walk through the building,
shooting at the targets until it reaches the other side.
Or it could clear a room with a default code and than just choose another
room, that it would breach and clear, until it cleared all of the rooms.
I don't know which would be more effective, but that's why I would
keep scores for the effectiveness.

But also, it could be possible, that a complex close quarters AI
isn't even needed, because there will be mostly big open areas,
meaning that the best version of the simple shooting range AI
should be fine.
And that would only prioritize finding cover and firing at the closest
target it can see, than moving to another cover where it can see the
rest of the targets from.

I always over complicate things.
I'm not working on a rogue like with infinite building variations.
I'll simply have a few buildings and that means, that I could just simply
make a path that the AI would move on and clear a building for every layout.
It wouldn't take too long and would work with ANY building I would later add,
because the AI would already know how the system works.
This would be the easiest with the best outcome, that would feel like the
AI are real soldiers tactically moving through a building and would be able to handle
any obstacle from, locked doors, that need to be breached, to heavy resistance,
where they would have to use grenades to clear out the enemy without massive casualties.



Why do I keep writing essays in the comments?

[andycrosby]

interesting video jonas, thanks for posting it. i really liked how your visualization showed the neural net firing as the action progressed. i agree that with more opportunities to train over time and a methodology for more random trainees you might get a more diverse array of strategies to emerge. you didn't really explain why you finally settled on the network configuration you used, it would be interesting to hear more about that. do other configurations possibly extend the solutions you can get, maybe more propagation layers or nodes are necessary? is it just a matter of computational power too? .. i recall reading a paper once about making the number of layers of nodes and connections one of the variables that it is randomized in subsequent generations.... and as jason kennedy pointed out with genetic algorithms testing multiple populations against each other gives you a sort of parallel optimization of different candidates instead of just a single champion each time. anyway keep up the good work very cool

[MichaelPohoreski]

You may want to color code the input / process / output neurons such as RGB.

[realinfinux]

I just started to use neuronal Networks and ask myself why did you use this specific structure and not just a normal feed forward one? Are there any advantages 🤔

[siddharth-]

Wonderful. Had a lot of fun watching it. I don't have a lot of idea but I think experimenting with the learning rate should do the trick. The funniest part was the one line AI(Can it be called an AI?) :)

[maralundo]

Hey Jonas, what if it escapes? Have you heard about Skynet?

[StarChargerStudios]

The problem is your having it face itself, which is why it’s narrowed down into that certain meta game

[thespontaneoustomato]

0:43
"Teaching *AN AN* AI to beat me at my own game"

[BrainSlugs]

If you'r neural network *literally* only has weights (and not biases, and no non-linear activation function(s)) then it will not be able to learn an effective strategy. (It may learn a little, but will hit a glass ceiling). Look at Sebastian League's video on the topic, the visuals on the video demonstrate the necessity of these two items rather well. Also, don't write your own DNN code. Use Unity ML Agents. 😉

[fraser]

If you worked on optimizing this to the point where you can simulate populations, and not just agents, more generalized and resilient strategies should win out.