Note – this was an article I wrote for the now Defunct GameSauce magazine. I quite liked the article since there’s so much meat in it, so I thought I’d reproduce it here. If you’ve ever wondered how making CGI movies is similar – or different – from making games, here it is.
We’ve all seen articles write and people say “Well, the games industry is like the movie industry, just in the 1930’s!” And in some ways that’s true. But in most ways? Not even close. The games industry – what it produces, how it makes that product, team makeup, distribution channels, ways of monetizing – it’s all very different.
But having said, that, in certain areas – that of CGI movies (pure, or almost pure computer generated movies – think Polar Express, Kung Fu Panda, Ice Age, Shrek, all the Pixar movies, and, of course, Avatar), there are more parallels than you might think. There are usually 2 different types of ‘pure’ CG movies – the totally animated (Pixar, Kung Fu Panda, Shrek, Rango etc) and the Motion Captured – (TinTin, Polar Express, Mars Needs Moms and so on). Then there are the non pure CG movies, which has some mix of Mocap, Live action and pure animated (Avatar, Real Steel etc). Most of what I’m going to be talking about are the MoCaped movies, since that’s where my experience lies.
But first, let’s do some background and talk about the process by which these movies are made.
Traditional movies are made by a production company – the start out with pre-production where the production company hires grips, camera men, set designers, costume designers, lighting gaffers, sound guys, make up people, actors, assistant producers, assistant directors, actors, visual effects houses and so on. From an established script, they design and build sets, build story boards, scout out locations, hire cameras, lights, sound equipment, rent stage time and so on. Then you get into full production when everyone shows up, the actors do their thing in front of the cameras, directors thrown coffee cups at interns etc. After that, you go into post production, where the film is edited together, music is scored and laid down, visual effects are generated and then the movie is delivered. Sounds simple but it’s a huge amount of work.
However a CGI movie is made in a different way, in a different order even.
The production company is the same, but who they are hire is different. You don’t need camera men (well, you do, but for totally different reasons), lighting equipment, locations and all the rest of it for a CGI movie, and what could be considered ‘real’ production starts much earlier. Digital set construction (low res versions of the sets) is started almost immediately – a sort of virtual story boarding, where the director hires a Pre-Visualization house to generate shot / environment / model ideas. Once the actors are hired and motion capture stage time is sorted out, the actual capture begins.
One of the biggest draws of CGI movies is the fact that you aren’t dependent on lighting or location or any of the million and one things that delays traditional movie production – assuming you set up production correctly, you can shoot in 4 weeks what takes 4-5 months on a traditional movie shoot because you are never waiting to go from one location to another (it should be noted that some directors do NOT like to shoot this way – James Cameron for example – he likes to complete a shot before moving on which elongates the shooting process but still makes for the same duration of movie development), or for the lighting to be just right, or for makeup to finish what they are doing etc. Robert Zemeckis (Christmas Carol, Monster House, Polar Express) has a timer on his stage – if more than 15 minutes elapses between shots, then someone is getting fired.
Once the raw mocap data is gotten, the resulting character animation gets integrated into the digital sets that have been created (and this is an involved process – the director has to ‘select’ which performance capture he wants for any given shot – much like an editor deciding which take to use for traditional movie making, although for CGI the director can mix and match performances from different takes which makes this a bit more complicated for the Motion Editor, who then puts all the performances together in one scene ), and at that point the director “shoots” the movie – deciding where the camera is positioned and moves for each shot. While he was on stage getting the mocap performance, some directors are not even thinking about where the camera goes (for the most part – movies that have CG elements mixed with live action elements E.g. Men in Black, Avatar, Transformers – you cannot help but be cognizant of the camera position, and some directors just like to experiment on set with potential camera positions just because that’s the more traditional way of Movie Making.) – he does that later, once the entire scene has been integrated together, ie the motion capture performance has been retargeted from actors to the final characters, placed in the set correctly etc.
After he’s done that, the movie is ‘rendered’ out – in incredibly basic form, think rendering tech back in 1994 – so the team gets an idea of how the movie looks, where the cameras are, and so on. This is then taken by editorial who ‘cut’ the movie together (shots are cut up, interspersed; some elements of the story are removed altogether etc). At this point the movie is complete as a watchable experience, but it looks somewhat like something rendered out using a Game Boy Advance as the rendering engine! This process is called many things – production Viz, directors layout – just like games and terminology, movies haven’t quite standardized on their terminology for this situation either.
So off this goes to the facilities house who will take these individual shots – along with everything that was used to generate it, – assets, animations, set designs, camera paths, particle effects, textures, models, the whole kit and kabodle – and then rebuild them using hi-res assets, high fidelity renderers and all the latest tech to produce the actual shots you see when you go to the movies. Sometimes this facilities house is internal – Sony has one, Pixar has one (obviously), ILM has one etc. But mostly these facilities houses are external studios – Weta, Digital Domain, The FrameStore, Cinesite, The Mill etc. In fact it’s not unusual for a movie to be made by several Visual Effects Houses – each taking a specific set of shots and turning them around. Visual effects for mixed movies (movies with real actors AND lots of CGI – think Star Trek) are usually made this way. The beauty of the CGI generated movie is that ONLY that which has been deemed as necessary to an individual shot is pushed out for the expensive bit – that of HiDef shot creation.
So theoretically it should be cheaper to make CGI movies – production is faster, less people are required on set (although the back end is literally hundreds of animators, CG effects content creators etc) and so on, however it’s usually not. Why is this? Several reasons. Firstly, the pipelines for developing these kinds of movies are built in reactionary mode – they aren’t planned, they are built in reaction to “Oh god, we need to handle this as well! Quick, write me something!” which doesn’t make for the most robust – or fast – pipelines. Established pipelines (like those at Pixar, Sony and Disney) tend to be a bit more stable and established, but even so, a game developer would look at them and see them as one special edge case process after another. Being mature doesn’t always mean you are doing things the right or most effective way, it just means you know where the rickety parts are and how to get around the bits that don’t work very well. Secondly, creating all this art costs time and money – we all know how expensive it is to make art for a game like Call of Duty. Imagine doing that at much higher res and visual fidelity, using expensive contractors to do it (everyone in the movie industry is on short term contract, so they charge more for the privilege of being let go once the movie is done)! And lastly, all that rendering take time and very expensive software and computer technology to do it.
How movie development is NOT like games
But still – it all sounds fairly straight forward right? Very similar, in fact to how we make games. And in broad strokes, it is. But like most things, the devil is in the details. Broadly speaking at a more 10,000 foot level, the movie industry has several crucial but subtle differences.
- Everyone is on contract. Production companies tend to exist for the duration of one movie (like everything Hollywood, this is a tax dodge) – they rent everything because they don’t want to have to sell it again once the company shuts down. Because of this, everyone is on short term contract. By definition that makes everyone far more expensive. The implication of this is that pipelines for producing CGI movies don’t ever grow – they are what they are and are the same every time a movie is made, because no one sticks around to evolve and refine the pipeline. You are hired for the duration of the production because that’s the way production companies have done it for years.
- Which leads neatly into the fact that Hollywood hasn’t quite woken up to what and how software engineering and asset creators fit in. With in Visual Effects houses, they have a little more idea – but even then they are considered instantly replaceable and temporary hires. Within the actual production cycle of a movie, they are even more in limbo. An example would be unions – you don’t traditionally get health benefits in Hollywood since you are usually on short term contract – so you have to provide those yourself. Usually your union handles that – you pay your dues and they provide health benefits. However, there is no specific union for software engineers in Hollywood. There is a visual effects union but almost no one belongs to it!
- Hollywood doesn’t tend to revise itself very often – most of the time it uses predictable and proven practices where the end result is known. This is because set time is so expensive – you have to be SURE you are getting what you’ll need at the end of the day because remounting a scene is VERY expensive. Given that, you use what you know works. But that acts against revision and new, unproven, technology. Which means that Hollywood tends to use fairly stagnant methods and processes. Every now and again, some large event happens and the pipelines get revised so everyone can do what him over there just made $500m doing (right now we are in the middle of such an event – the use of 3D in movies) – but these aren’t the norm. They tend to happen every 20 years or so. But because Hollywood *isn’t* reinventing the wheel per movie, as we tend to in games, they *can* operate on the contract methodology because experience on Movie A is just as relevant on Movie B – it’s not a completely different engine with all the attendant gotchas you either know or you don’t.
- Ideas are King. In the games industry, ideas are two a penny – everyone has one. Implementation – along with a decent, wide enough idea – is everything. Who thought that Rare could make such a great James Bond game? It wasn’t the license that made Goldeneye what it was, it was implementation. However in Hollywood, ideas are everything. What’s even more different is that a production company will quite happily invest and form a team to make a movie based on one person’s idea, who isn’t even employed by them. Frequently I have people in the movie industry say to me “I have this idea for a game, who do I go pitch it to?” and then I have to explain that, no, the games industry doesn’t work that way. Unless you have a game team ready to make that game – and preferably a prototype – pitching that to Activision will get a polite “thank you” and you shown the door.
- Hollywood is VERY good at squeezing money from a movie. They have multiple revenue streams from a movie – first run at the theaters, second run (the dollar movie theaters), pay per view, airplanes, hotels, then the DVD release. And that’s not counting the same overseas. With far more distribution than we get in the games industry. How many Xbox 360’s do you think there are in India? There are way more people who go to the movies, let me tell you! J Plus, they also don’t have the problem of shelf life in terms of “well, it won’t run on today’s consoles”. Who wants to play Halo 1? Can you even if you want to? A game traditionally has a much shorter shelf life and no legs after the first year at all. Contrast that with the fact that you can go buy Citizen Kane on bluray tomorrow if you want to.
However, these are broad strokes and not specific to CGI movies but they do play into how Hollywood is not really the same as game development. More specifically there are differences for the CGI production aspect – for example –
- Assets tend to get created twice (or even more times on occasion). Now, that does also happen with game development too, particularly early assets in a situation where the engine is having features added to it, or file formats upgraded. But in general it’s not something that is business as usual. It is for CGI movies.
What’s worse is that often the hi-rez version of the assets are never re-used for the sequel. Given that the hi-rez assets are often developed by a 3rd party VFX house, if that VFX house isn’t on board for the sequel then those assets just sit on a server, un-used by anyone. However, even if the VFX IS brought on, chances are that rendering hardware will have moved on, and the assets will need to be reworked to handle wet hair, or increased polygon budgets etc.
And, of course, most of the assets are rigged in such a way as to be specific for the pipeline that the VFX house has in place. Face rigs are a notorious example of this. - Assets for movies tend to be MUCH higher resolution. Video game artists spend a lot of time taking 20,000 polygon models – of a human figure for example – and creating normal mapped 2000 versions of them, for increased rendering speed. Movie assets tend to start at 200,000 polys and are then further sub-divided by the off line renderer. Movie assets ( at least human figures) don’t tend to bother with things like normal maps, since the resolution is such that crease effects are part of the model, not needing to be painted on later (it is worth mentioning that environmental models still use some of these techniques, just that anything that comes anywhere near a close up will not).
- Everything is offline. All the main rendering for lighting and so on is done in separate passes and then ‘composited’ together at the end of the pipeline. Unlike real time renderers, where everything is done in 1/60th of a second (hopefully) and all sorts of tricks are used to get a result that looks pretty good. Movies aren’t really interested in ‘pretty good’ – they want as accurate as possible and can spend time and money throwing hardware at the problem. Shots are broken down into passes – particles, characters, lighting, cast shadows, contact shadows and so on – that are then pushed together into one frame, with motion blur, depth of field, grain and color correction applied over the top (and what’s more, each step is a specialty within the movie industry – a lighter is NOT a compositor for example). For the technically minded, this is the equivalent of a deferred renderer, with each process becoming a shader pass – just all done offline.
Now having said all that, the pipelines and processes that Hollywood uses to create CGI shots and movies *is* broadly the same. However there are areas where Hollywood could actually use being more like game development. For example :-
- The afore mentioned re-useable assets. Hollywood needs to own all assets used in creating a movie so they can be used again, or even shared between VFX houses (currently they are not – each VFX will create its own version of an asset based off a low res version the production house develops). The reason they don’t do this already is just because of either the short term existence of the production company, or because if you have 3 sets of VFX assets because you use 3 VFX companies, where do you store these? Storage costs money and chances are they won’t get reused down the line anyway.
- The concept of ongoing pipelines – Most VFX houses lurch from one movie to the next (in actual fact the VFX business is notoriously hard financially. Many houses go to the wall because the amount of profit made from doing VFX is very low – the margins just aren’t there because of the number of VFX houses bidding against one another) and spend very little time rebuilding pipelines or replacing the duct tape that’s applied to get Transformers 4 : More Bronzed Limbs out the door. Game development houses generally DO spend time investing in their tools and pipelines – they’ve learned that with each console generation they HAVE to, or they risk being an also ran. Movies have not learned this yet. It’s the tools stupid – Iteration time is king.
- Better and more real time tools. One of the things that game dev has while production is underway is access and use of the game engine. Those titles that have accurate preview of asset creation, scripting, level building and other disciplines within the final game engine are by far the most polished and fun. This way you get fast iteration and get to see what the final result actually looks like (and what it might do to frame rates) as you are building them. Movies do not have this capability – so much time is lost in building a scene that takes 4 hours to render before you can even see what you just changed.
It’s made worse because of the terrible rendering facilities at the disposal of the pre-viz team – often the director, when he’s shooting the movie, has a lack of lights, no depth of field, no motion blur, a very limited polygon budget (lots of movies do pre-viz with no textures, to speed up real time rendering for example) and so on. Often the shots that are produced have way too much vagueness in them when handed to the VFX house, who basically fill in the blanks with their best guess, and if they get it wrong, well the director shouts at people and the shot is redone – mostly at the VFX house expense.
If movies had a decent game engine renderer to produce somewhat accurate visuals, but in real time, quite a lot of money (in the millions of dollars) could be saved. What a shame no one is working on this. *cough*. - Synergy between game developers making “movie titles” and the production companies they are working for. LucasArts and ILM had a ‘Synergy’ project they tried implementing a while ago, with the aim in mind of bridging between the two companies. The idea was to share assets between movies and the games being built on them. Now this is not a new idea – production companies have often ‘thrown over the wall” movie assets at developers (Ubisofts Avatar used all the pre-viz assets created by Lightstorm), however most of these assets are often a) not created for games and b) specific to movies. The animations created for Star Wars : Attack of the Clones are very specific for individual shots and aren’t something you can just plug into a game. So the idea was for the movie guys to get a better idea of what games needed, and create a little more when they were building their animations / assets for movies so they *did* just slot into a game, plus also develop tools to allow sharing of asset definitions – for example, both ILM and LucasArts would use the same particle generation tool – the only difference would be the back end. In the movies, particles would be tiny and there’d be millions of them. In the game, larger and less of them – but both would operate against the same physical rules, so the effect in the game would look and behave like the movie particle effect, just at a lower resolution.
There’s quite a lot of this that *could* be done, but just isn’t because so many VFX houses and CGI movie production companies are so in reaction mode and not in forward thinking mode. The VFX house that does do this will find itself much sought after by production companies who are also thinking of their IP being a game.
So there are parallels between movie making and games, just not as many as you’d think because, as I said earlier, the devil is in the details and with making movies, the details are everything. However, it’s a really cool place to be, there are a lot of places to shine and what’s occurring now in software / tools development in movies is going to dictate how movies are made for the next 30 years. Plus there are Academy awards, lots of cool parties and hotties waiting to be ‘discovered’. Now… where did I put that business card that said “producer” on it…?