The Murky world of Mocap

In this post I wanted to talk a little about the mocap process and its use across various productions. The process appears to be becoming a very attractive option to studio’s looking for quick results and a realistic looking motion. However the process is a lot more technically demanding and animator driven than some sources would have us believe.

In this breakdown of a single frame from the film, we can see quite clearly some of the processes and animation layers involved in creating the final performance we see on screen. In the first of the five production stills, we see Zoe Saldana’s original performance as Neytiri, clearly visible is the head rig that is recording the facial movement that will later drive the initial facial animations.

The next image along is the CG ‘template’ and is the real time representation of Saldana’s performance displayed during the motion capture shoot. In this image we can see that the facial tracking is close but does not completely correspond to the live performance as Cameron had previously claimed. Although this is a lower resolution model, key attributes such as the eye line of the character appear off and the whole expression feels somewhat subdued in comparison to the original performance.

It is at the third image that we finally start to see the input of the animator; this image shows the first animation pass in which the facial capture data has been applied to the higher resolution model for animating. The animator’s job mainly consisted of making sure that the expressions matched that of Saldana, assisted by the HD reference footage shot during the take. In this scenario, motion capture was able to help in cutting production times by having the basics of the motion ready to be enhanced and refined rather than reproducing each frame from scratch, whilst ensuring it remained true to the original performance by closely matching the reference footage.

The next panel along depicts the next stage of the animation and effects process whereby the animation is applied to another higher resolution model and the hair and cloth animations tested and checked to ensure that they reacted to the motion appropriately. This process would include checking that the hair did not intersect the head at any stage, that is to say that it did not pass through the model unrealistically and behaved according to the physics of the world.

The fifth and final panel shows the final rendered still from the movie. This matches Saldana’s expression in the still from the motion capture shoot but is a vast improvement on the source motion depicted in the second panel. Motion capture has definitely played a part in helping build the animation and in turn save some time, however it could be argued that the reason the digital performance is so true to the original live action one is through the additive animation created at Weta. In returning to the source motion and handkeying sections, the animators were able to add in all of the subtle nuances that made the original performance believable but were too minute to capture, and in doing so, assist in reproducing the mannerisms of the performer. When speaking about the facial animation Cameron eventually admitted that in allowing the animators to work in a more traditional manner, i.e. keyframing rather than relying solely on their skilful manipulation of the motion capture data, that ‘The animator recognised the details that were missing and put them in. It was that last 10% that made it real’

With a high rate of productions employing these techniques it would seem reasonable to suggest that people need to be educated better with regards to the work that goes into getting motion capture looking sharp and refining the performances to the levels we see on screen. If actors want to become a champion for the cause, please credit all parties involved, the writers, directors, vfx artists and animators rather than being so self-centred. Yes the actor deserves credit for creating the base performance upon which the animator works from – creating a powerful performance in an environment with no real props to speak of, in a skin tight suit with a large head rig but still able to convey that emotion, just wow. Looking at it from this perspective I can totally understand where actors are coming from, their work is phenomenal but as James Cameron says in the making of avatar – it’s a performance that’s been turbocharged by the animation dept! – The only real mention any animator seems to get with regards to dealing with motion capture. As the various software packages used to clean mocap data state it is the BASE layer of the performance.