A. I. Learns to Play Starcraft 2 (Reinforcement Learning)

Tinkering with reinforcement learning via Stable Baselines 3 and Starcraft 2.

#artificialintelligence #machinelearning #python

[serta]

I have to say you make the most understandable learning materials
Your website together to the videos. All the Code is there, the book, the playlists from scratch. Most professional educators can’t do this 🤗

[fuba]

Very interesting idea with a macro ai and a strategic ai, sort of working in tandem forming a symbiotic relationship of sorts.. could maybe even break that down even further, like on a per unit type basis... Tho i imagine the complexity explodes at that point.

[sebbes]

The "ghosts" could "fade" over time, but never fade to zero again (caped at minimum 1, starting at like 255 or something), to make the algorithm prioritize the most recent ghost locations.
Also, instead of scouting with void rays, wouldn't it be cheaper to scout with drones (to generate ghost areas) (scouting probably targets mineral areas without ghost, to see if enemies have expanded, while voidrays can scout areas WITH ghosts, to see if the enemies are still there & try to defeat them there, can also send a probe first to ghost area, to determine enemy strength before attacking).

[Derrekito]

Never before has a marketing ploy worked so well on me. I'm looking forward to receiving the hardcover version of the book!

[kailalueni]

I love you idea of drawing your own minimap! Thats a smart way to make more information available easily.

[fuba]

This was an interesting video. I will have a look at your example code for sure, wanna try to tinker a bit. Thanx for all your hard work.

[JohnJackKeane]

I do not code or have the desire to code, but this video is beautiful. I enjoy StarCraft videos seeing people micromanage, but the thought and process that goes into creating a “program” to do the same thing is fascinating. The amount of work and work to obtain the knowledge that goes into the work is far underrated. I hope for you the best!

[adityachawla]

Here is an idea: You can use more then 3 channels to give spatial information to your network. No need to limit yourself by conventional idea of 3 channels! If you are worried about how to visualize this, just think of it as an extra map.

[kevintyrrell]

14:49 That's some next-level Gateway placement.

[awsamalmughrabi]

I like how in depth this video is, really enjoyed it!

[VaSoapman]

Why not give rewards based on how many enemy units/buildings are destroyed?
Then give a penalty based on how many units/buildings are destroyed?
Also to help the AI prioritize winning over stalling, you could increase the value of a win based on how fast it won.

[adye]

This is freaking intense! also for the hunters problem: Why not make a "return to safe space" function for them when they detect enemies. That way they only perform scouting duties.

[protoplmz]

Hey! I love the update here. I followed the original series you put out. As a SC2 veteran I noticed deficiencies and deviated in a strong way halfway through. I setup separate models to handle the decision making for each aspect of the game. This makes it so it can make the decision to use its army separately from the decision of progressing tech (or not). I stopped around the time I couldn't figure out how to have it build its own strategies as I ended giving it a long set of possible actions and letting it pick and it felt too 'guided'. It was able to beat "Very Hard" 50% of the time vs random's 0%.

Was my first exercise with ML. I got the chance to apply the concept it at work for something outside of my scope. Used both that and the SC2 project as demonstration in an interview and got a promotion out of it. This inspires me to try my hand at it again!

EDIT: To handle army movement which you mentioned in the video, I chopped the maps up into a grid and gave it decisions to make where it could attack-move its army to any of these at will. 9 worked the best but you could make it much more granular. It used this to both attack and defend.

[faithful]

I'd love to see the next video in this series with dual macro and micro algorithms and improving the win percentage

[SocalNewsOne]

Thanks! Your tutorials were the first that worked for me. Biggest problem that I had was the directory path for the Starcraft maps.

[XmKevinChen]

It’s a very interesting video about the ML + gaming. As a newbie to this AI world, it also gives lots incentives to continue learning.

[EnderSword]

Kind of neat, I'm wondering if you looked at the AlphaStar research at all to do this, or looked into the StarCraft 2 AI community? There's about 70 coders of various bots and AI that compete against each other and it'd give you a ton of ideas on build choices and especially unit control and decision making.

[Neceros]

This is great! I'd love to see something like this could compete in the arena

[nastrimarcello]

This amazing. Amazing code, amazing explanation, amazing editing.

Only one suggestion: when possible, don't use try:...except:pass
As this can lead to hellish problems.

If you know what exception you are having in that try-except statement, using that exception explicitly is better (even if you are just going to 'pass' it)

[pognar]

I have played starcraft for years and years, and I love this channel.
This is going to be great.