Warren Spector recently gave an interview to Eurogamer that’s circulating pretty widely now about the need for better AI in games. Specifically what he’d like to see is people like John Carmack and Tim Sweeney stop focusing on graphics and start working on creating believable characters and immersive worlds.
Of course, I’ve got some issues with this. These guys are graphics programmers, they aren’t AI experts. Thinking that a graphics guy can just up sticks and refocus on AI is effectively part of the cultural issue AI has – it isn’t considered a serious discipline, it isn’t “hard”, it isn’t something people are experts in already, so let’s call in some graphics programmers to solve the issue! Of course, this is completely untrue and shows a bit of contempt for the AI that does exist and the experts who are working on it right now. We don’t need John Carmack to solve AI for us, we need people to listen to the Carmack-esque characters who already exist in AI and who are ignored because “players don’t want good AI”.
However, this minor piece of bitterness aside, what Warren is talking about makes a lot of sense, and it’s something I’ve been saying a lot over the past couple of months as part of a lecture tour that’s taken me to South America, Europe and across the USA. One of the key themes I’ve been talking about is AI outreach and advocacy – helping to explain what AI is and how we can use it to make better games. So, because Warren has set the stage so well by bringing it up, here is the article version of that presentation!
Why So Scared?
AI is seen as a really scary topic by a lot of people. I think a big part of it is that we’ve been culturally conditioned to see intelligent machines as an inherently bad thing. We’ve had decades of books and movies dedicated to the enslavement of humanity, if not an outright apocalypse, all because of robots and AI systems getting a little too smart for our own good. It makes people kind of wary. But it doesn’t have to be that way! All AI really boils down to is the applied science of decisions. There are whole branches of maths dedicated to analysis of decisions, Decision Theory being an obvious one, but also Game Theory and even Economics. These all allow us to theorise about how we should react in a given situation, and after we do all the algorithmic analysis, we end up with a neat result on a whiteboard and we walk away – problem solved. AI is about putting that solution into practice, whether it be in some simulated environment or with a robot.
Why is AI scary? Well, this guy has been our spokesman for nearly 20 years…
But the big question is, where do we see these kinds of decisions? Typically when gamers talk about AI in the context of games you might hear that “The AI did such-and-such”, and what they really mean is the enemy in the game. There are an awful lot of decisions happening there, it takes a surprising amount of sophistication to create characters that can believably run into the player’s field of view and convincingly fall over when killed. And whilst this is what a lot of Game AI focuses on as a discipline – what you might call opponent AI – there’s a lot more possible. People often say that players don’t want better AI in games, this is what they’re talking about. And to an extent they’re right, because what they mean is that players don’t want opponents that can out-think them. Part of my advocacy has been to address this specifically, especially in the academic community where “better” typically means stronger and more capable. If that was truly the goal of Game AI, as Blizzard’s Schwab pointed out at GDC ‘11, we could just make NPCs that were faster, stronger and had more powerful attacks. Making a game more challenging in and of itself isn’t hard. Academics – especially AI academics – like to talk in terms of optimal solutions, so one of the breakthroughs I’ve had is to explain Game AI as AI that tries to optimise “fun”. Some people get it, others then want to pontificate about an algorithmic definition of fun, but that doesn’t alter that as a core concept, this is what we are trying to achieve by using AI in games. What I want to do though is explain to people how we can use AI to do this. I’m not going to deal in technical details, because what I want to do is explain this in a way that designers and producers can follow, and implementations already exist out there.
The Best Game Ever Made
I want to start out by asserting that Dungeons and Dragons could quite easily be described as the best game ever made. It, and the table-top genre as a whole, has endured in a way no video game has to date, and its supporters aren’t growing tired of it, it’s just a bit of a pain to organise compared to firing up World of Warcraft. The great thing about D&D is that it’s not a game itself, it’s a rules framework in which games can be created and players aren’t restricted in what they can do. Of course that means that players will try to do weird and wonderful things, things that could never be predicted beforehand and that’s where D&D can come into its own. The thing that makes this possible is the human Dungeon Master who controls the game. The Dungeon Master is part of the game, and part of the gamer group. Before the game starts they create the world and build up an idea of what the plot of the game will be. It’s important to note that since the players drive a lot of the game, the DM can’t plan out the entire storyline in advance, because as any good strategist knows, “No battle plan survives first contact with the enemy”, and players need to be able to shape the story that is unfolding in their own way. Invariably however, this unfolding process will lead to a combat encounter, which requires a map to be created and populated with creatures for the players to battle. There may be plot devices as part of the encounter or specific elements that foreshadow something to come later which combine to create a subtly cohesive experience for the player. Once the encounter is created and the players are engaged in it, the DM will take responsibility for controlling whatever creatures have been placed into the map. This is a fairly interesting part of the role, because the DM has to balance the need to take believable actions that would make sense for those creatures against the need to make an exciting – and not too challenging – experience for the players. If the players will die from one more hit from a goblin archer, it doesn’t make sense for the archer to turn and fire into the wall. This isn’t a believable action though, and players in general won’t like that there is only an illusion of risk. Schwab described this as “Losing with style”, and it’s a key aspect to this kind of player experience management – we don’t want to be blatantly throwing the game in favour of the player, but at the same time, we don’t want to be insensitive to the fact that they are playing a game and don’t want to be frustrated or have the game end prematurely. One of the most important ways to manage this is to have a good understanding of who you are running the game for, knowing what types of player are in the party and how they will react in certain situations. This allows the DM to have a much better handle on how to stage manage the game to get the desired outcome, as well as tailoring the content to suit particular player types – if you know that players in your group tend to favour finding diplomatic solutions rather than confrontation, you can ensure that the options for this kind of play are available. Equally, if you have a player who is liable to charge at every band of enemies presented, you might like to prepare “surprises” for that player to exploit that they’re going to do something foolhardy.
The Dungeon Master is, in my opinion, a significant part of what makes D&D and games like it so successful. They have a longevity completely unmatched by any other games from their era, and continue to enjoy popularity and a zealous fan-base. To my mind, they represent the “Gold Standard” of what we should be trying to emulate when we make games today – not necessarily in terms of setting and tone, but certainly the design philosophy of empowering players to make choices and have real, tangible impact on the worlds we are playing in.
Of course, the obvious way to do this would be to ship a DM with every game, but I’ve been assured that cramming guys into retail boxes specifically to look over the player’s shoulder and tweak the game in real time would be considered a human rights violation. …Hippies. So instead, what we typically do is pre-bake a lot of the DM’s job during development. We fix the whole plot of the game upfront and reduce it to a sequence of combat encounters stitched together with cutscenes. We play those encounters out in a set of fixed levels, crafted by a designer once and then never changed, no matter how many times a player works through them. And then, we spend the vast majority of our time focusing on controlling the minions within the encounters – to such an extent that this almost becomes the totality of the wider perception of what Game AI is.
Of course, it is so much more than that so I want to close out by discussing briefly how AI can be used to start to replicate the DM’s role, and hopefully create powerful, engaging experiences as a result.
AI For Storytelling
This one is a bit over my head, but we can use AI to dynamically tell interesting stories, to take into account the player’s actions and adapt the story to take this into consideration. We’ve started to see this becoming increasingly mainstream lately, with Bioware being one of the big pioneers of some of the lowest-hanging fruit here with branching narrative. This is more of a 70’s era expert system than what we think of as true AI, but consider a game such as Mass Effect, where your choices at one part of the game are going to influence what happens and what choices are offered later. At a basic level you might think of this as a choose your own adventure book, but even that rapidly becomes pretty complex. Now add the nuances that Mass Effect and the like introduce into dialogue, recurring characters and so on, and the combinatorial explosion on the potential state graph is pretty intense. We can go a stage further though, because not only can we represent an enumeration of potential states using AI techniques to effectively manage this, but we can start to introduce actions within the world that are the AI system steering the player towards certain states in this notional graph. If we specifically want the player to arrive at a particular point of the story, we can push them back in that direction and begin to constrain their choices. If done carefully, we can give the illusion of complete choice, whilst still stage managing the experience. We can also build in concepts from literature to this, and manage the dramatic pacing of the story by modifying how components are shown to the player.
Choosing your own path through an entire trilogy proved to be so engaging to players, that they nearly revolted when this choice was removed.
There’s a strong, and growing, Interactive Fiction movement that has some great resources on this subject. A great place to start if you are able is Marc Cavazza who has a lot of publications centred on the theme.
AI For Encounter Design
Procedural Content Generation is one of the latest crazes sweeping through the game development community. At its most basic, it says that creating content, whether that be quests or a varied array of weapons or even levels for the game, is a time consuming process. For a certain amount of overhead, we can instead create an AI process that will make that content for us, and make as much of it as we need. Skyrim demonstrated this effectively with the Radiant quest system, which enabled the developers to create a seemingly limitless range of side-missions without ever having to write one of them by hand. The system could pick, in a mix-and-match style, a location for the quest, a task, a target and then assign it to the player. Tiny Wings is another great example, where every day the levels the player experiences change – without a way of generating these levels algorithmically, it wouldn’t be feasible to do this since it would be so time consuming.
No discussion of PCG is complete without a Tiny Wings reference!
At its heart, procedural content is about creating things dynamically, and may seem to be random, but rarely is. Randomness in general is a bad idea when you are talking about replacing a talented content creator such as a level designer – there’s a reason these folks get paid to make content, and it’s not because they can be replaced by glorified dice. With that said, it’s often possible to capture something of their insight and build that into the generation system, to create the kinds of content that they would. Using Tiny Wings again, each level has a different parametric description, so the overall feel of the level is kept the same and the difficulty progression varies in the same way each time, despite the specific details changing day-to-day.
Procedural generation is something that’s really close to my heart right now as myself and Heather Decker-Davis have been relying on it heavily for a casual title we’ve been working on, and in turn it has shaped a lot of our workflow and tools. You can hear more about our experiences and some tips we shared for working with procedural methods in gave a great overview of their approach last year.
AI For Character Control
As I said earlier, this is traditionally an area that gets an awful lot of attention, so I’m going to gloss over it a little bit, except to say that there are a broad range of techniques that you can use to create characters, such as Behaviour Trees, Goal-Oriented Action Planners, Hierarchical Task Networks and even Finite State Machines. Ultimately though, you need to understand what you are trying to achieve – is it to provide the player with a challenging opponent, or an engaging companion? The AI techniques required to die convincingly are separate from those required to create a believable set of responses to the weird and wonderful things that the player might decide to do. If you intend to make opponents, you need to consider if they work in squads, and whether a squad is a loose collection of affiliated enemies, or a well-trained group following specific orders, perhaps from an officer within the group. The Halo franchise demonstrated that small squads with officers could implement a morale system effectively and have squad cohesion disintegrate with the death of the officer.
These design choices are going to have a direct impact on what AI techniques are going to be most appropriate to create the desired feel to the characters in our game worlds, and it can be some of the most immersion breaking moments for players when the characters don’t act the way they expect; they are willing to believe that an alien swarm has taken over New York and the government has ordered the military to kill everyone involved, but they aren’t willing to believe that it’s possible to kill one of a pair of soldiers without having the other react in some way to his companion’s death.
There are lots of resources for reading more about this kind of AI, from AIGameDev.com through to a large portion of the AI Summit held at GDC, available through the Vault. The AI Game Programming Wisdom series of textbooks is also a good source of material, and when it comes to understanding the rationale behind the decisions characters might make in specific circumstances I also highly recommend “Predictably Irrational” by Dan Ariely, which is a book about so called “Behavioural Economics”, or where decisions meet psychology head on.
AI For Player Modelling
Player modeling in games is a relatively new approach that has yet to see wide adoption, but which has the potential to be a bit of a game changer, especially when combined with some of the other techniques already discussed. At its most basic, player modelling is something we’re already pretty familiar with whenever we play a game with humans; we try at some level to get a sense for who they are and how they play. This is most often seen in games like Poker where the stereotypical classifications of players, such as “Tight” and “Loose” have become part of our vernacular. In these competitive games, we use this kind of classification to predict an opponent’s behaviour and exploit this, but again this isn’t really the aim of Game AI – sure it’s great when an enemy opponent adapts to your tactics and challenges you to try harder, but we can use this technique for so much more than this.
Silent Hill: Shattered Memories is a great example of player modeling in action. In the opening scene of the game, you find yourself in a doctor’s office undergoing a battery of psychological tests. Unknown to the player however, the scene is set up in such a way that although the character is being given these tests, really the player’s responses to them is what matters. In truth, the testing is being done on the player, and it’s actually a standard psychological profile being built up. Based on this, the player is put into a category, and throughout the game, different content is shown to them dependent on their category. It doesn’t change the overall story of the game, but it does change the tone and feel of the experience.
This is the same scene, as experienced by two different types of player
But what happens when you don’t have the luxury of running a complete personality test on the player as the game begins? Let’s face it, games would rapidly begin to feel pretty dull if they all had to find a justification for doing this! It turns out that just as in Poker we can start to classify players as we observe their playstyle, we can do the same thing with other games. In 2009, Drachen et al at ITU Copenhagen were able to analyse data from playthroughs of the AAA title Tomb Raider: Underworld. What they found was that by applying some machine learning approaches, they could extract a number of stereotypes of players that people would broadly fall into, namely “Veterans”, “Solvers”, “Pacifists” and “Runners”. This was based entirely on ingame analytics, the kind of data we all have access to already.
Using AI systems then, we can learn a lot more about our players and how they interact with the game. But we can also feed that back into our content generation or narrative systems to start customising the experience directly to specific groups of individuals. This is something we’re seeing explored in the academic world through such things as the Mario Level Generation competition that has been running for the last couple of years. It’s something that we’re starting to see come to “real” games a bit, and in particular it’s something that myself and Heather have been working to put into our indie title. For us, it’s pretty much a no-brainer. Our game is based around a simple maze system, and this is broadly what our PCG system creates. If we can detect players who are interested in exploring the maze and ensuring they have collected everything before progressing, then we can adapt our content to prefer to give them more sprawling mazes with a wider range of things to collect. Equally, if we find players are more liable to dart straight to the maze exit and take more risks with the hazards in the maze, then we adapt to suit those tastes too and provide a more action-oriented experience. As with the human DM, understanding the players tastes and desires means that we can cater more directly to specific individuals and get better engagement with them. That means they have a better experience, more fun and means we’re making better games!
The End
I really hope that this post has helped you to understand a little bit more what AI can do for your games – in a very real way, Warren Spector is bang on the money, AI has the potential to be the next big thing in game development but we need to get more designers who aren’t intimidated by it! If you want to learn more, there are some great resources online that can help you get started, and of course every year the AI Game Programmers Guild hosts the AI Summit at GDC which is a great source of more inspiration. And who knows, maybe you’ll run into Warren there!