This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
When designing a tools pipeline for your game, you could do worse than to adopt The Unix Philosophy as stated by Douglas McIlroy.
One way to interpret this philosophy is to separate each step of the pipeline into a single command line application, which takes a input file (or set of files) and produces an output file (or set of files). Running your export pipeline then is a case of defining a script which calls each executable in turn, passing the output of one as the input of the next. You can write a GUI that is used to set up any options parsed to the individual steps to provide an interface to the user and as each executable will be writing out data your GUI can pull that data from stdout and display any warnings or errors that were generated along the way.
![](im4ges/Unixpipeline.png.scaled500-300x227.png" width="500" height="379"/></a>
</p>
<p>Perhaps the polar opposite to this approach is the MegaEditor™. This is the application that does everything, a sort of Excel for games development. It is a tempting and attractive option, the MegaEditor™, the user has everything in one application; you can unify the UI; and the there is just a single codebase to maintain. However, this approach is simply not as flexible as multiple command line tools.</p>
<p>Imagine this common situation: there is change in your model format and all the models in your game need to be re-exported to run in the latest engine.</p>
<p>With the monolithic single application your users would either have to load up each model and hit the export button, one at a time. Or more likely you would have to implement a new feature to the tool, with at least a minimal GUI, to allow batch exporting of a set of models.</p>
<p>With the separate tool pipeline approach you can write a script in a couple of minutes that takes a list of models and runs them quickly through your pipeline as a batch process. If it becomes clear that this is a feature that will be used frequently by users then it is no problem to add the GUI part at a later date, which provides a front end to the batch script.</p>
<p>Taken as a whole, implementing that feature fully in both cases is basically the same work. Implement a GUI option, call your export code multiple times for many model files. The difference is that in the second case, with the simple command line tools, we can get a batch export running in under 5 minutes, then if needed, we can add a GUI later on. With the monolithic app the GUI is a required part of the tool and the coder who writes the new feature must be familiar with codebase in order to implement the feature. When you’re in a situation where none of your models work with your game engine and your whole development has ground to a halt, setting up a batch export in under 5 minutes is invaluable. ( Of course if you’ve managed to get into a situation where someone has changed the model format and brought your development to a halt then you should probably have a think about how your team operates, but still… )</p>
<p>In addition to this flexibility with individual tools you get a built in modularity to your code. If you’ve decided to re-write your texture processor, simply keep the input/output file interface the same and you can drop in the replacement very quickly. Decided you need an extra step to your pipeline, after the collision export but before the final compile? Write your new tool and drop it in between those two. Best of all, none of these tasks requires touching the code relating to the other tasks, so your changes will be more stable.</p>
<p>Talking of stability having discrete executables has another advantage in that they are inherently testable. There is no need for complex unit testing frameworks to maintain. You can automatically test your tools, by running them on a known asset and checking that the result is as expected ( e.g. with a binary diff ). You can then be sure that after making changes you haven’t inadvertently broken something.</p>
<p>So to summarise, build your pipeline with the Unix Philosophy in mind and you get:</p>
<ul>
<li>flexibility</li>
<li>modularity</li>
<li>testability</li>
</ul>
<p><span style="http://twitter.com/daredevildave">@daredevildave</a> on twitter.</em></span></p>
<p><span style="http://codeamplifier.com/blog">blog</a>.</em></span></p>
<div class="nr_clear"></div>
<div id="nrelate_related_1" class="nrelate nrelate_related nrelate_default nr_text"></div>
<!--[if IE 6]>
<script type="text/javascript">jQuery(){kind=link}