Why is It Difficult to Make Good AI for Games?

Having made over 100 videos digging into how AI works in video games, I've never really explained why this is such a big problem. Let's discuss some core AI fundamentals.

[00:00] Intro
[00:34] Playing the Game Vs Being the Game
[03:21] Understanding State Spaces
[06:51] The Challenge of Good AI for Games
[10:55] Why Not Use Machine Learning?
[12:58] Closing
[14:32] Credits

Be sure to catch the rest of the AI 101 series to learn the foundations of AI for games.

--
AI and Games is a YouTube series on research and applications of Artificial Intelligence in video games.

Follow 'AI and Games' for educational videos and essays:

My second channel 'AI and Games Plus' is dedicated to bonus content, deep-dive analysis, revisiting old topics and more:

Listen to my podcast 'Branching Factor', as I discuss game development and research with my co-hosts and guests:

--
You can support AI and Games via Patreon, YouTube memberships, and Substack In return, you get bonus rewards!

--
Follow 'AI and Games' on social media.

--
If you're interested in working with us, check out the following:

[AIandGames]

Let me know what topics that pop up in this video you'd like to see more of. I'm keen to do videos digging into some of these fundamental questions, but also dig into those projects in games like Overcooked, Street Fighter and Rocket League.

[Xxnightwalk]

I'd say the hardest part for me so far has been making AI fail in a fun and credible way
I often say game AI has to be "Smartly dumb", and that is hard to do

[goosewithagibus]

I was working on an AI for a turn based rpg I made that would take whatever weapons, spells, and team composition to factor in how each enemy took their turn. I also gave them random traits that were invisible to the player such as "cowardly". This would make them behave in a super predictable manor for me, the developer, because I could follow a logic tree and know exactly what they're going to do, but it was hard for my friends to spot the patterns because of the sheer number of combinations. Definitely a lot to think about but it was super fun. I scrapped the project for the time being but I'll probably come back and rework that AI at some point.

[JockeSohls]

I have been working as a Combat/Game Designer with a heavy focus on Combat and AI for around 15 years at the point of writing, and I think this was an excellent video! I will for sure look at more videos on this channel!

I also wanted to echo something that was brought up in the video that I always talk about when covering this subject: people do not necessarily want smart AI, they want AI that makes themselves feel smart! The illusion of smart AI is far more important than them actually being smart, and if the Player has ample tools to predict and outsmart their enemies, that will make for a very good experience.
I also always like to say that making smart AI is super easy, but making a good AI that is fun to play against is super hard.

[davesparks]

I've been mostly designing and modding for an existing franchise with very little effort put into the AI. So I decided one day to build/implement my own systems into it from the ground up as a learning experience and way to improve the game for others. FSMs, Behavior Tree, GOAP, Utility AI, etc.. I built a framework for them all. I find the challenging part isn't building the systems. In fact, they are all surprisingly easy to create. It's designing within them that is difficult. For example, even a simple behavior tree can break down (mostly in the 'this no longer makes sense and looks incredibly dumb in this situation' way). With my limited experience in mind, I can say the biggest challenge I come across is making what normally looks intelligent not look incredibly stupid when something unusual happens.

[fuseinabowl]

The biggest issue I had was lack of familiarity - while researching ML applications for a preprod game I had to work very hard (and was ultimately unsuccessful) in investigating and assuaging concerns about ML. We were all familiar with existing techniques for AI and in AAA environments so you need to find the places where ML is superior - triggering a voice line when the player enters a room is not improved by adding ML, but balancing a complicated army composition might be. Even in the cases where ML has some advantages you normally need to give something up (typically designer control) and that's compounded by our negativity bias. If I were to do it again I'd make a more structured experiment and agree upon targets beforehand, making sure to involve people who were sceptical about it.

[dirge]

With the focus of improvements/advancements made to visuals, I feel like Ai has long since taken a backseat to any sort of focus, that modern Ai in games feels like it hasn't changed much since the 2000's.

I really do wish devs would dedicate some time to Ai in the future, because I'm honestly getting put off modern games with how deadpan and lifeless AI act.

[Draliseth]

Ya know, one of the biggest gripes about a game like Breakpoint that a lot of people have is the enemy ai. However if you look at a game like Phantom Pain, especially after a game like Breakpoint, it seems like heaven. That said, there's nothing about the artificial intelligence being smarter that makes it better. What I think people want more than outright tougher enemies (as in smarter, more tactical) without realizing it is a breadth of behavior. Even in Breakpoint's immediate predecessor, enemies had more behaviors than unaware/alert/killmode. In Wildlands You could catch some sleeping or doing pushups to pass the time, things like this. In Phantom Pain you had direct interactions that made them seem more intelligent such as being able to hold them up. Sometimes, they'd go for their gun. Other times, you could tell them to kiss the floor and it was effectively a knock out until alarms went off. That said, I don't think developers give enough time or mind for those seemingly rare emergent moments that make a game shine for the people who discover it.

I cite Breakpoint because immediately as I played it, I got whiffs of other games like Red Dead Redemption 2 but none of the substance (bivoacs, rations, etc. being surface level mechanics).

A lot of people talk about ai in terms of difficulty. I think the better, less frustrating option is ai presentint variety to allow for emerging gameplay.

Sorry, was too busy writing that to pay attention. I'm gonna rewind and watch now so I can see you address that exact thing. 😄

P.S.
I also had Starfield in mind as Bethesda seemed to catch a lot of crap for dumbing down enemies in the game, but I honestly think it was a good idea, especially for the space fights as those really are difficult until you level things up and would bottleneck most players until they stopped.

[daveface]

This is a great video from a technical perspective - the performance constraints placed on AI are definitely a large factor, e.g. instead of any kind of box/shape trace, Half Life 2 uses line traces for enemy vision. It's faster, but it also means you can hold a brick in just the right spot to block the trace and become invisible.

But design intent plays a significant part as well, because even if a programmer could design the smartest AI that always picked the optimal strategy, that would make for an awful gameplay experience. How do you distinguish between AI that has interpreted realistic inputs and come up with a clever solution, from one that is simply 'cheating'? As a player, you can't - and even if you could it still wouldn't be a fun experience.

The example I tend to think of (I'm sure others can think of better scenarios) is a stealth game where you're chased by a bunch of enemies into a room with no obvious way out. You move some crates and find a hidden vent, then pat yourself on the back for being smart. You start going through the vent, and just as you're nearing the exit - an enemy pops up and lobs a grenade at you. In a multiplayer game, if a player did that then they might have known there was an alternate path you'd try to use - so they moved to the vent and timed the grenade just right. You'd (maybe) cogratulate them for outsmarting you. But when you know that it's an AI, your first thought isn't that the AI followed that thought process and outplayed you, you're going to think the game was just tracking you through the wall or there was a scripted event that threw the grenade. Even if you did think it was a very clever AI, what's the fun in this situation? Most games wouldn't work if every enemy was as smart as the best possible player of the game, we rely on some amount of stupidity for the power fantasy to work.

[frankeeeej]

I'm currently making a fully fledged 3d soulslike.
My current AI-script boils down to:
"If you see the player, run towards him, maybe circlestrafe a bit, and attack"
It's not exactly complicated, but it works really well actually :d

[LyingTube]

I had a pretty bold but straight-forward concept: Player actions would create portals that, after a certain gameplay point, would open and release an invasive hive-type NPC faction to colonise the area around the portals.

The problem: The world was procedurally generated and the player could change almost every part of it. I spent weeks trying to parse how to interpret any conceivable arbitrary collection of positions into a set of "rooms" that the NPCs could assign functions to (barracks, farm, storage, etc.). The functions would then request an NPC be present so they could operate, essentially as a supervisor while the room itself ticked over. Ultimately, I realised that I could just group the positions into the largest contiguous cuboid available, remove the points it contained from the set, and repeat until the size of the cuboid was too small to be useful.

Weeks of staring at the problem, and I finally realised I could just treat the points like a series of cubes, in a voxel game. 🤦

[StephaneBura]

Great video!
Another issue is the need for game designers to be AI-litterate so that
a) descriptions of the features that will be handled by AI can be as clear as possible and the amount of interpretation by the programmers minimal,
b) the designers can imagine new ways of exploiting AI's possibilities.
AI programming in complex games is not something that occurs once the design is done but a constant conversation toward a unified vision. Without a shared language, this conversation cannot occur.

[JosephCatrambone]

12:00 - I'm pretty sure you know this, but for others that don't: the purpose of Alpha-Star wasn't ultimately to make an ML to beat StarCraft. The reason StarCraft and others are a good learning experience (ha) for ML researchers is because there are certain problems where we really don't know what intermediate states lead to a good outcome, but we do know a good outcome when we see it. For folding proteins, we don't necessarily know which intermediate states are the correct ones, but we DO know the resultant energy of the final folded state. For two chess boards, you might not know which one is better, but you definitely know if someone is checkmated. And for 11:00, one other detail is that machine learning doesn't always give the appropriate control over the behavior of the AI. You might want the AI to sometimes be less aggressive or less accurate or run away. With a classical system you get that fine-grained ability to decide what the bots do; not as much with AI. It's also easy for them to exploit weird quirks of the game if they're trained with reinforcement learning.

[michaelmaguire]

Because no one actually wants "good" AI, they want "just barely good enough" AI. (Which is why MLA controlled enemies would be a nightmare unless crippled.) I remember back in the day hearing from a couple of different game devs that they ended up having to make their AI worse, because it proved too difficult for people in play-testing.

[anthonyapm]

A video on the ai of Black and White would be awesome.

[ruano]

The chief problem is not whether we can build good AI; it's that we can't really define what is good AI. Or we can't come to a consensus. If the goal is simple like it needs to be really competitive and good at defeating the player then actually that is generally fairly easy to achieve. Give the AI a version of aimbot in a shooter and it will kill the player almost every single time. But we don't want that because it's not fun. Most players don't necessarily want a very challenging AI as if they were competing against real people, they want something more casual; less intense. If good AI is one that is good at competing with the player then you might be seeing games where the best you can hope for is a 50:50 win rate against the AI and if you're not great at the game maybe you are losing 90% of the time. That game will instantly stop being fun. So I think AI is often intentionally bad because it's fun for the player to feel like they are great and can win most of the time.

If good AI is to replicate human behaviour then again, I suspect most of us won't like it. Mainly because we like the AI to be predictable so we feel like we are learning over time how to defeat it. If it's very human like then it will either be a bit random in its behaviour and we end up raging against the 'RNG' or it gets better and better over time and we end up finding it too challenging. Besides, human players do all sorts of stupid things that if an AI actually replicated it we would instantly complain the AI is stupid. E.g. think of all the times you were playing a game and accidentally fell off a cliff, forgot to activate a skill at the right time, just screwed up the input on your controller etc. If you saw the AI do that, you won't think, 'wow, that's just like a real player'. You'll think, 'that is a stupid AI'.

Even something as simple as pathfinding is not straight forward. If you want the AI to just get from A to B fastest it would do what some human players do and jump over obstacles, sprint everywhere etc. But in a game world we would find that weird and think the AI is behaving unrealistically. We want the AI to walk around predictably and respecting all the roads and social conventions even though we as players often do not respect them.

So at the end of the day the developers have to produce an AI that is not too efficient, not too good, follow rules that we as players have no intention of following, and make us feel like we are great at the game. This is for the mass market anyway where the money is. The only solution I can see? A somewhat 'dumb' AI.

[DeyvsonMoutinhoCaliman]

Some of the hardest games of the past barely had any AI other than repetitive movements. A truly intelligent AI would be undefeated, it would simply charge toward you with all the enemies together instead of letting you take them down one by one, or a few at a time.

[esnevip]

I'd love for you to cover the subject of personalised opponent AI, similar to the drivetars.

One of the biggest hurdles in fighting games is finding someone who is near your skill level, I hope one day we can have a personalised ai opponent that learns how you fight, and punishes your bad habits, and always stays just that TINY bit better than you so you can grow.

Edit:typo.

[marcusm]

I made a video many years ago about evolution videogames where characters change over time to be best suited to the game environment. It doesn't use deep learning but has some results

[sempersolus]

Thank you, Dr. T.
They did not cover this at my Uni. I think my Comp Sci department was more of a mobile app dev mill.