The Importance of Everything: Analytics of Map Design

In this 2014 GDC session, Epic Games' Jim Brown covers some of the successes and failures of level design metrics and analytics, and show how they can be used improve your design process.

GDC talks cover a range of developmental topics including game design, programming, audio, visual arts, business management, production, online games, and much more. We post a fresh GDC video every weekday. Subscribe to the channel to stay on top of regular updates, and check out GDC Vault for thousands of more in-depth talks from our archives.

Follow us on Twitter

Check out our Facebook page for GDC exclusives

[Adalore]

And this is why I cannot take devs citing "Our statistics" unless they write a thesis talking about HOW the statistics actually mean anything.

A very good talk.

[BologneyT]

I'm not even a pro game designer and I learned more about the world from this video than I did from an entire year of college.
One hour well spent.

[smort]

"You see a pie chart and your brain thinks of pac man, so you end up chasing ghosts."
*Smoooth*

[junpeiiori]

One of the best GDC talks I ever watched. Kudos to Mr. Jim Brown!
And thanks to GDC for sharing this enlightening session.

[hanzflackshnack]

"That armband concept is cool but it seems creepy when he puts it as watching everything you do for analysis" *Looks at phone.... "Oh...."

[briandecker]

Wow. This is easily one of the the best GDC talks posted so far.

[PHeMoX]

Hmm, fairly good talk when it comes to the basics and some of the pitfalls, but it's mind-blowing how poor general level design in games actually is these days. Most modern maps fail instantly when leaving the concept of mirroring the lay-out behind, as asynchronous level design needs way more balancing. Even worse, a lot of modern games seem to have maps that *have no design to them*, but rather are stitched together art assets. It's great when a map is designed with visuals in mind almost from the very start (as really most levels are if they strive to become any kind of realistic or familiar setting), but I don't for a second believe enough time is being spend thinking about a map's potential in the phase where the design is still reviewed in the top-down overview!! That is, when the design is still at the drawing board phase and can be balanced purely in a spatial manner. (Most of the least favourite maps do look horrific when just looking at them in top-down overview).

With that in mind, I tend to disagree with an inverted approach to level design where player statistics define basic level design decisions afterwards. When a map shows such a high deviation of what was expected in terms of player spread, then it's probably a flawed design from the very start.
The boom shot weapon example is a perfect example of how that information of dying players has little to nothing to do with the level design itself. The weapon definitely creates a motivation for the player to go to said weapon locations, but when comparing corridors to a larger open spaces, you'll always see a matching set of statistics. Assuming a level has no additional 'motivators' for players to go elsewhere, it's not unlikely for large amounts of players to end up in the larger open space or in corridors towards them.

And lets be honest, it's a bit dumb to change core designs based on changing statistics indefinitely. It should be quite possible to get an equal spread of activities throughout a level; when a level has a certain 'motivator' on one side of the level, you'd better make sure the motivator on the opposite part of the level is in balance with it or you'll start to see people concentrate around one area (much) more. I'm also pretty sure sometimes player behaviour alone can change how effective the original intend of a map was (think rocket jumping in old Quake, something that later was embraced in level design).

The problem is more in how games through modern progression systems have left the idea of pick-ups being super important, yet at the same time lack weapon balancing to make things truly 'fair'. The upcoming Quake Champions seems to try to change this by having a mix of default load-outs and pickup weapons (which are better more powerful versions), but honestly they probably should not have a weapon load-out system to begin with. And I'm not convinced their weapons currently are balanced along the entire spectrum anyway.

I could probably write a book about how weapon design *and balance* is a thing closely related to both level design and the fun and challenge of playing a game. I'm a big fan of games that also have a proper weapon balance between all available weapons, or indeed encourage such a play-style difference you will find some people have a strong preference using a certain weapon (within a certain areas of the level) over others. (Think flakgun in UT'99 that encouraging a rushing very offensive play style in corridors, versus a bio rifle that works way better when you let people chase you down creating hazards for them to pass / dodge a fully charged bio goo shot in their face for an instant kill also in corridors or for example the sniper rifle in a more open space). Not many games will have a good statistical balance when it comes to weapons vs weapons and all manner of stats like 'time spend dueling/hits delivered/damaged done/reload time/charge time for secondary fire/game mechanical advantage (shotgun spread of bullets vs single bullet high damage-slow-reload sniper rifle) etc.'. It should be fairly easy to use proper math to balance those from ground up.

Also, the 'green games' about 10 minutes in (a now outdated list from 2011, I know) all had massive marketing behind them, good timing of their releases and substantial hype amongst customers, because in reality it features quite a LOT of games that actually aren't objectively all that good at all, let alone really better than many of the games in the following 20.
I do not for a second agree all of those games in the top 10 had 'great level design' or dealt with statistics the proper way. Or even kept players busy for longer. I actually think specific developer loyalty from the fanbase and the fact this top 10 is filled with basically nothing but sequels is really very telling. (8 unique IPs outside of the top 10, versus basically only 1 in the top 10 (which is really kind of a GTA-with-horses anyway)). And again, not all of the sequels in the top 10 were close to flawless games in what they were. I played a lot of GTA IV, but I liked Sleeping Dogs way better in terms of level design and gameplay. I'll admit it had plenty of flaws of it's own. But it really should have sold way better than it did.

[iamisandisnt]

OK so I think I agree completely. The only problem I have with data analysis is that people are unpredictable. I don't wanna show up at Disney and have the same exact experience catered for me that I had before - just like how I do use Spotify's suggested music selection, but it's a love/hate relationship. They know the right songs to play, but it's always the same damn songs.

[ZarbanDerek]

That wasn't two different problems. They solved the SAME problem two different ways: first by moving the weapon and second by making the new location of the weapon hard to defend. Other ways they could have solved the problem: randomize the weapon's location; make the weapon run out of ammo and scatter ammo around the map; make the weapon run out of ammo and then respawn fully loaded elsewhere.

[nikolaisafronov]

Good practice is to give two people one set of Data and two different Hypothesis and watch what who will get a better case

[zaneseibert]

"I feel pretty safe in saying those guys will break that pattern with Titanfall." *Cue the Price is Right loser jingle*

[TheLeontheking]

watching that guy with the red shirt in the right is just as interesting as the talk itself

[Scarabola]

Perry Mason is instead being remade by HBO. Looks like they didn't get the memo.

[simonjohnwright]

So thats why Netflix makes such terrible recommendation of a load of B-movies I never want to watch. If only they put Perry Mason on instead or anything that might help build a better future society. If I like Commando it does not mean I like action films, nor does it mean I like Arnold, or the Director or anything else. I might like the narrative of Commando and the FX of Star Wars, and the main character of Scent of a Woman. From watching those 3 films what would Netflix recommend? The problem is trying to be predictive, just make good stuff in the first place instead of trying to please the masses. MGS, Socom, Counter Strike, Space Invaders, Pacman, Sonic, will always be good games. There is no formula to making a good game. If you are a good developer then you just know the difference between a good or rubbish game and please stop believing in six sigma or any of the other bs, your best creativity comes from yourself.

[tOmzzvideo]

Data and information are not the same thing, data becomes information once given meaning. That aside it was a good talk!

[SuperEssenceOfficial]

Yo i got it i knew it was Tim Sweeney 9:00

[hanzflackshnack]

College age women have the highest percentage of gamers? There's no part of my brain that can comprehend that being true

[moffetdamacy]

great talk, but this guy has a particular cadence to his "public speaking mode" that he comes back to at least once every 30 seconds and its driving me nuts.

[refundreplay]

No, thanks. I wouldn't like to "magic band" my games and thats creepy as fuck. good god.