One problem which often arises during programming is how to build a base set of functionality which can be extended by the user, while still being modular enough to make it easy to replace only certain parts of an implementation without having to resort to copy & paste techniques. I guess everybody of us has faced this problem at least once, and came up with different solutions. There is a powerful and elegant technique called policy-based design for solving this kind of problem, which is what I want to show today by applying it to a mechanism common in game development: logging.
The problem
Let us assume that we want to add logging facilities to our game, and for that purpose we build a simple base class called Logger, which can be extended (or completely replaced) simply by deriving from it and overriding a virtual function. Note that we are not concerned about how log messages are dispatched to the different logger implementations, but rather how new logger-classes with completely different functionality are implemented.
A simple base class for loggers might look like the following:
class Logger { public: virtual ~Logger(void) {} virtual void Log(size_t channel, size_t type, size_t verbosity, const SourceInfo& SourceInfo, const char* format, va_list args) = 0; };
As you can see, it’s nothing more than a very simple base class with one virtual function. The arguments are the log-channel (e.g. “TextureManager”, “SoundEngine”, “Memory”, etc), the type of log (e.g. INFO, WARNING, ERROR, FATAL), the verbosity level, a wrapper around source-code information called SourceInfo (file name, function name, line number, etc.), and last but not least the message itself in terms of a format string and a variable number of arguments. Nothing spectacular so far.
One possible logger implementation could be the following:
class IdeLogger : public Logger { public: virtual void Log(/*arguments omitted for brevity*/) { // format message // do additional filtering based on channel, verbosity, etc. // output to IDE/debugger } };
The IdeLogger outputs all log messages to e.g. the MSVC output window by using OutputDebugString(). In order to do that, it formats the message in a certain kind of way, applies additional filtering based on the channel, verbosity, etc., and finally outputs the message. We might want another logger which logs to the console, and one which writes into a file, so we could simply add two additional classes called ConsoleLogger and FileLogger, which both derive from the Logger base class.
Sooner or later, this is where we run into problems, depending on what we want:
- A logger which writes to the console, but only filters based on the channel, not the verbosity level.
- A logger which writes into a file, but doesn’t filter any messages because they are a useful tool for post-mortem debugging.
- Slightly different formatting in one of the existing loggers, e.g. I like being able to click on log messages in Visual Studio’s output window because they are formatted like this:
“C:/MyFilename.cpp(20): [TextureManager] (WARNING) Whatever.” - A logger sending messages over a TCP socket, without having to copy existing code for formatting/filtering.
- Many more such features…
A deceivingly simple solution
One solution which is sometimes applied to such problems is to put certain parts of an implementation into several base classes, and build a deep hierarchy of classes by using multiple inheritance. In essence, you end up with classes like ConsoleLoggerWithVerbosityFilter, FileLoggerWithoutFilter and TcpLoggerWithExtendedFormatting which multiply inherit from concrete implementations of base classes like ILogFilter, ILogDestination and ILogFormat. Personally, I tend to favor flat hierarchies without any coupling over deep hierarchies where leaf classes are sometimes affected by changes to some base class.
Furthermore, judging from experience you often need to add a bunch of virtual functions to such hierarchies only to make some leaf class work, or end up copy-pasting existing code because a seemingly innocent change might break some of the existing implementations, which is why I try to stay away from such solutions – they might work for certain problems, but I’ve often seen them break during the development of a product.
Anatomy of the problem at hand
Let us take another look at the problem, this time from a different angle, leaving out implementation details like base classes, inheritance, and so on.
What we essentially want is a mechanism which makes it easy to define new loggers based on existing functionality without copying any code, and without having to write a new logger implementation each and every time. What we want is a mechanism where parts of the implementation could essentially be assembled by writing only a few lines of code.
By splitting the logger’s responsibilities into smaller pieces, we can hopefully find some orthogonal functionality along the way. A logger essentially:
- filters the incoming message based on certain criteria, then
- formats the message in a certain way, and finally
- writes the formatted message to a certain output.
If you think about it, these aspects are completely orthogonal to each other, which means that you can exchange the algorithm for filtering messages with any other without having to touch either the formatting or writing stage, and vice versa.
What we now would like to have is some mechanism for exchanging those aspects with very little amount of code. Such a mechanism can be achieved by making use of templates, like in the following example:
template <class FilterPolicy, class FormatPolicy, class WritePolicy> class LoggerImpl : public Logger { public: virtual void Log(/*arguments omitted for brevity*/) { // pseudo code-ish... if (m_filter.Filter(certain critera)) { m_formatter.Format(some buffer, criteria); m_writer.Write(buffer); } } private: FilterPolicy m_filter; FormatPolicy m_formatter; WritePolicy m_writer; };
The above is a very generic logger which passes on the tasks of filtering, formatting and writing messages to certain policies, which are handed down to the implementation via template parameters. Each policy does only a very small amount of work, but by combining them you can come up with several different logger implementations with only a single line of code, by means of a simple typedef, as shown later.
Example policies
Before we can discuss the pros/cons of this approach, let us quickly identify what some of the policies might look like in order to gain a better understanding of how such a system works:
Filter policies
struct NoFilterPolicy { bool Filter(const Criteria& criteria) { // no filter at all return true; } }; struct VerbosityFilterPolicy { bool Filter(const Criteria& criteria) { // filter based on verbosity } }; struct ChannelFilterPolicy { bool Filter(const Criteria& criteria) { // filter based on channel } };
As you can see, each policy takes care of filtering messages based on certain criteria, nothing more.
Format policies
struct SimpleFormatPolicy { void Format(const Buffer& buffer, const Criteria& criteria) { // simple format, e.g. "[TextureManager] the log message"; } }; struct ExtendedFormatPolicy { void Format(const Buffer& buffer, const Criteria& criteria) { // extended format, e.g. "filename.cpp(10): [TextureManager] (INFO) the log message"; } };
Writer policies
struct IdeWriterPolicy { void Write(const Buffer& buffer) { // output to the IDE } }; struct ConsoleWriterPolicy { void Write(const Buffer& buffer) { // output to the console } }; struct FileWriterPolicy { void Write(const Buffer& buffer) { // write into a file } };
Discussion
By dissecting the problem into different aspects, we can now implement very small functions/structs called policies, which can be assembled together in any way we wish by using just a single line of code. Some examples:
typedef LoggerImpl<NoFilterPolicy, ExtendedFormatPolicy, IdeWriterPolicy> IdeLogger; typedef LoggerImpl<VerbosityFilterPolicy, SimpleFormatPolicy, ConsoleWriterPolicy> ConsoleLogger; typedef LoggerImpl<NoFilterPolicy, SimpleFormatPolicy, FileWriterPolicy> FileLogger;
Advantages
- With only 3 filter policies, 2 format policies, and 3 writer policies we are able to come up with 3*2*3 = 18 different implementations, just by using a simple typedef.
- Each part of an implementation can be replaced separately, so if you e.g. add a TcpWriterPolicy you can combine it with any other filter or format policy, without having to resort to copy-pasting.
- Pieces of policies can be assembled in any way we want.
- New policies can be build using existing policies simply by combining them.
- If any combination of policies does not exactly do what you want, you can still implement your own logger by deriving from the Logger base class, without being forced into a certain inheritance structure, except the single virtual function in the base class.
- Simple, flat hierarchies, no multiple inheritance used.
- One virtual function call instead of several ones (multiple inheritance of several interface classes).
- Unit testing becomes a lot more easier because each policy has exactly one responsibility.
An example of how to build new policies out of existing ones:
template <class Policy1, class Policy2> struct CompositeFilterPolicy { bool Filter(const Criteria& criteria) { return (m_policy1.Filter(criteria) && m_policy2.Filter(criteria)); } private: Policy1 m_policy1; Policy2 m_policy2; }; typedef LoggerImpl<CompositeFilterPolicy<VerbosityFilterPolicy, ChannelFilterPolicy>, ExtendedFormatPolicy, IdeWriterPolicy> FilteredIdeLogger;
Drawbacks
- Template parameters are part of the class name, leading to longer class names, in turn making the code harder to debug if you’re not used to it.
- Increased compile times if you’re not careful. I would recommend using explicit template instantiation for required logger implementations.
Conclusion
Policy-based design is a powerful tool which can be used in several situations, not just the one shown here. It can lead to extendable and modular classes if applied correctly, but it isn’t a silver bullet which can forever solve our architectural problems, hence it shouldn’t be applied blindly.