Andy Firth is an Engineer Architect @ Bungie in Seattle. He's been working in games for 13 years mostly in low level Engine / Rendering related roles. Most recently he's been concentrating on infrastructure and workflow related problems especially where concurrency and multi-platform comes into play.
Code Build Optimisation Part 3
So we’ve all done it. You’re coding away and you want to call say, a math function from your templated utility class that lives in your header. So you #include its header in your header, everything works allowing you to move on. It’s a common pattern that many engineering projects are purely feature/result driven; write the code, debug the code, check in and you’re done… next task. Rinse and repeat this pattern for a few months/years across many engineers and you inevitably find that your include structure is heavy & costly. But it all works right? Who cares that it takes 5x longer to compile now. If you care, read on.
Once you get to this stage it is often very difficult to figure out what is going on. Depending on the age of your code base you can often be looking at an include tree 4-5 levels deep (sometimes far worse) and involving 1000′s of files. How do you retroactively figure out where to direct your attention? Hopefully I can help you there.
I should note that the basic method here was written by one of Bungies Senior Engineers
An Example Analysis from Our Engine
The colorization in the above graph is intended to offer some context such as an SDK include, a windows header, an internal library etc. I prefer the DGML form because it is an active document allowing searching and hotlinking to code as well as graph analysis and reduction of focus. I output the graphviz form also due to DGML being a feature of VS2010 Ultimate only (~$10k per copy) so few engineers have it. The Graphviz form is a simple image and thus not AS powerful but still really useful.
Once this was up and running i wrote a set of VS Macros to provide direct access. From any file in the project an engineer is now able to single click and see badness rank and/or the include structure of their files.
Future work on this will involve augmenting the heuristic to include characteristics of the code within headers such as template usage or inline expansion both of which contribute significantly to compile times.
Hopefully others recreate the tool (it took only a few hours) and use it to help their projects & compile times.
enjoy.
