I’ve been thinking about motion controls quite a bit lately as a control scheme that I may have written off too quickly. I’ve also been obsessed with acting, body language and expression. And then it dawned on me. The two may be a perfect couple.
Games are made of verbs. Run, jump, punch, crouch. Hand held controllers are able to simulate the input of the player to use those verbs in a rather precise way, be it on/off or analog. And as developers, designing around verbs is a pretty straightforward affair which we have been doing since the first game was played. But when it gets to adverbs, the emotional variables of a verb, neither the controller or developer seems to be aware of what to do with them beyond pure visuals.
Enter motion controls. While initially it is the verbs we assign to the gestures, such as swing a sword, we quickly find out that they just don’t quite match up the crisply defined input of a controller. Motion Controls also lack a tactile sense of feedback in many cases, making all those verbs feel hollow when we don’t feel the physical reaction. It is a fun gimmick, but we quickly grow tired of the theatrics and plug our plastic hands back into the machine.
So if verbs aren’t best used for motion controls, how can we use them as adverbs and adjectives?
Some emotional poses are so universal, they can even be understood when displayed by inanimate objects
Body Language
Our bodies display our moods and emotions as much as our voice or face. How we sit, cross our legs, touch our face, or with what speed we move conveys a wealth of information about our true intentions. Recognizing those gestures and poses is something our brains process on an almost unconscious level. Like most things we do unconsciously, purposefully translating that can be rather hard to execute. But what if we added the recognition of body language into our games? For example, if someone is getting tired, the game could recognize it and scale back the pace. Maybe the player is getting bored, so the AI scales in difficulty. What if the player is displaying frustration? Possibly the game could provide a hint or item to help out.
Another approach could be adding body language controls to another player that essentially directs a number of influences onto the game. Much in the same vein as I’d proposed in a previous article, with the idea of another player using instrument controllers to conduct the tempo or player buffs of a first person shooter, how you choose to express emotions with your body could lead to all sorts of additive features to the game. An entire game around the first player as the traditional hero escorting the second player, whose only ability is to manipulate the world based on their body language could certainly lead to a deeper ludic connection of the player’s roles.
The first step towards any of this would most likely require some canned poses built into the system that the player can assume to tell the game what they are “feeling”. But if we could build the library of those poses to include some more natural ones, a real sense of personal connection could be built with the game. Your acting choices could be met with those of the game, opening up a true dialogue and conversation.
Suggestive Poses
If we are talking about the controlled use of gestures to illicit subconscious reactions in others, then there are certainly a few professions that we could adapt towards gameplay. From politicians and preachers to lawyers and prostitutes, all have figured out what gestures can be used to further suggest their message. Be it a game where you have to amp up your troops before battle, seduce an enemy into giving you information, give a convincing closing argument to a jury, or hold the attention of an audience as you give a speech, body language and gesture choices would make all the difference in the world. The easiest method of integrating this would be essentially quick time events, but if they were built in not as the sole method of defining the win/lose scenario of a game, but as a way to nudge the scales in your favor, it could be a powerful tool for both the player and developer.
Gestures can captivate an audience, no matter the message
At a certain point, players will certainly try to cheat the system, by either acting bored, or frustrated, as a sort of emotional “tilt” mechanism. This is great tho, as it would blur the line and intention between game and real life. Often times people try to act or emote in a manner that will get them what they want from others. Building an entire game system around the idea of faking emotions, in an attempt to trick the game or other players, could lead to some interesting scenarios. What if the player was a hostage or being tortured, and had to keep their captor distracted long enough for a rescue team to save them? What if this was added as a multiplayer component to an L.A.Noire style of interrogations, where you try to throw off the other player with your actions?
Cultural Poses
So beyond the obvious technical issues or reasons why I’m sure many of you are shaking your head at such propositions, let me heap on another issue. Localization of gestures could be rather tricky. Every culture not only has unique gestures or expressions but also vary to what degree they express themselves physically. If an entire games core mechanics revolved around body language or gestures, there is a good chance it could require a complete retooling when being localized. If the idea is to gauge how strongly you should show or press your emotions, those levels would need to be balanced completely differently for an American market vs a Chinese market.
BUT, we could also use those differences to our advantage. The chance to learn, practice, adapt or even exploit cultural gestures in a game space could be a fantastic social experience. How different cultures react to themselves or towards others is a game that many play everyday in real life. Being able to replicate or amplify that experience sounds incredibly fascinating to me as both a developer and player.
Other Input Methods
I don’t believe motion controls to be the only means towards making such experiences. Voice could be used in much the same way as I mentioned earlier, using pitch and volume as meters to create additive experiences. It certainly seemed like such an idea was being attempted in Milo, as I remember at one point in the video he questioned why the player was sad. But imagine adding those variables to voice commands when ordering squad mates. Depending on how stern or timid you instruct them affects how likely they are to succeed in their mission or even follow orders.
While it seems to have retreated into the shadows, the Wii’s Vitality Sensor certainly could mimic these same ideas. Touch controls, as far as pressure sensitivity, could certainly replicate some of these ideas as well.
I would also be remiss if I didn’t point out the idea of hugging or petting your pet in Kinectimals as a first step down this path. While those are both verbs, there is a strong emotional attachment and purpose to those actions. Silent Hill:Shattered Memories also had some interesting emotional effects attached the action of pushing off the monsters as your only means of dealing with attacks. Inherently it is a wild, thrashing motion, but the need of some precision in control forced the player to still keep some wits to them in an intentionally erratic scenario.
Closing Argument
I’ve only skimmed the surface on how body language, emotion, gesture controls and performance could all come together. Tying it in with my earlier post of the player and game as actors seems like a no brainer as well. But if any of these ideas excite you, I’ve some wonderful resources for you to absorb like a sponge.
If the topic of body language, especially as it relates to different cultures, is something that interests you, then Desmond Morris is going to be your new best friend. He has a The Human Animal, a series that ran on BBC, that will excite you to no end.
I’ve also been hooked (pun slightly intended) on the books and lectures of Acting for Animators is going to include how his principles relate to games. After having just attended one of his talks, his enthusiasm, appreciation and respect for this medium is something I haven’t seen before from others in the same vein.
Adapting their observations into games can be done in any number of ways, not just through motion control. But with motion controls being at a sort of cultural tipping point, it certainly seems like a relevant approach towards creating new experiences.