March 2017
Volume 32 Number 3
[Patterns]
Active Events: One Design Pattern Instead of a Dozen
By Thomas Hansen | March 2017
Editor’s Note
When I approached MSDN Magazine Senior Contributing Editor James McCaffrey to review the preliminary draft of this article, he came away fairly outraged by some of the opinions and ideas put forward by the author. Most days, that would spell curtains for an article draft. But as McCaffrey notes, it’s all too common for new ideas in software engineering to be dismissed simply because they’re new. And while McCaffrey says he’s still outraged by many of the statements in this article, we both agree that it may provoke a lot of thought about software design and methodology paradigms. —Michael Desmond
For as long as there’s been software, there’s been software that fails. According to studies, 25 percent of all projects fail completely, while 50 percent can be described as “challenged.” Imagine if a car manufacturer had the same statistics, and 25 percent of their cars wouldn’t start, or exploded when you turned on the lights, while 50 percent could do only 25 miles per hour or used 15 gallons of gasoline per mile. More important, what if customers wouldn’t even know which cars would work before they’d paid for theirs. This is pretty much the state of software development today.
Various solutions have been proposed to improve this state of affairs, none of which goes to the root of its cause—the code! In my opinion, the reason software projects fail can generally be reduced to the same problem: spaghetti code. And the reason the code is in such a sorry state is because there’s really no way to accommodate change. That’s why methodologies such as eXtreme Programming and Agile software development have been recommended. However, none of these can actually help create better code. Being Agile doesn’t magically help you avoid spaghetti code. To become Agile, what you need is to be able to separate your concerns into atomic building blocks.
What I’m presenting here isn’t an alternative to Agile or other such methodologies. Agile is, after all, a management framework, not an architectural design pattern. Instead, I’m proposing a design methodology that allows you to more easily become Agile because you can prepare up front for change. Over the past few years I’ve developed a design pattern I call “Active Events.” This design pattern lets you create small, reusable components that you can easily extend, modify, reuse and exchange with other parts. It allows you to think differently about software so you can “orchestrate” your code, almost as if the individual pieces were made of Legos. You may be thinking that this is pretty much what writing code using an object-oriented programming (OOP) paradigm is. However, Active Events is very much different than traditional OOP.
Check Your Own Code
Do me a favor: Open up the last solution you built in Visual Studio and check out your dependency graph. How many projects and classes can be extracted from your solution? How many classes could be reused in future projects? How many could you replace with other classes without breaking your software?
My guess is that your project and class dependencies look like they do in most other projects. It’s highly likely your project’s dependency graph resembles spaghetti. Sorry for being brutally honest, but that’s what my experiences indicate. But, relax, you’re not alone. Although I don’t have hard data, I would claim 98 percent of the world’s code suffers from the same problem. Besides, I wouldn’t force you to confront this issue unless I had a solution for you.
If you could somehow untangle those dependencies and create reusable components, such that none of the individual parts of your code reference any other parts, your programming life would improve substantially. If your projects and classes were not as entangled as they are today, you could more easily exchange individual pieces. If your classes were untangled, you could reuse them in future projects. If your projects held minimal dependencies, you’d have a range of components that are more likely reusable instead of a monolithic monster. Active Events facilitates zero dependencies across your classes and projects. When it comes to software entanglement, zero is the only acceptable number!
How Do Active Events Work?
The Active Events design pattern is actually surprisingly easy to understand. Instead of creating interfaces to separate the implementation and consumer of a piece of logic, you simply mark your methods with an attribute. Here’s an example:
[ActiveEvent (Name = “foo.bar”)]
protected static void foo_bar(ApplicationContext context, ActiveEventArgs e)
{
var input = e.Args[“input”].Value as string;
e.Args.Add(“output”, “Hello there ” + var);
}
This syntax is taken from Phosphorus Five, an open source Active Event implementation I created. See “Software That Almost Doesn’t Exist” for more information.
To invoke a method that has been defined using the Active Event paradigm is equally simple. Instead of invoking your method directly, you indirectly invoke it through its Active Event name:
/* ... some method in another assembly ... */
var args = new Node ();
args.Add ("input", "John Doe");
/* context is your ApplicationContext instance, which keeps
track of your Active Events */
var result = context.Raise ("foo.bar", args)["output"].Value as string;
The Raise invocation invokes the foo.bar Active Event. Note that there are zero dependencies between where you invoke your Active Event and its implementation. In addition, there are no common classes shared between them. Only data is passed in and out of the events. Also notice that all Active Events have the exact same signature. Hence, classes, interfaces, and methods move a bit behind the scene and become less a focus in the source code.
The loading of the Active Event components is completely dynamic and can be done either through a list of assemblies referenced in your application’s configuration file or by automatically loading all DLLs from a folder and registering them as Active Event handlers. Internally the Active Event “kernel” will traverse all types in your DLL during loading and store a MethodInfo reference to its Active Event methods in a string/MethodInfo dictionary. This dictionary is later used during invocation of your event. If you wish to scope an Active Event or associate state with it, you can dynamically add an instance of a class as an Active Event listener and use an instance method instead of a static method as your Active Event sink, which is what’s being done in, for instance, the plugins/p5.web project in Phosphorus Five.
This creates an environment where your different parts are simple building blocks, invoking each other indirectly, which allows you to easily exchange any single Active Event assembly with another implementation. Completely changing the resulting application is as easy as changing a configuration setting or replacing one DLL with another. And it’s just as easy to, if you wish, change the string “foo.bar” to “foo.bar-2.” At the same time, each assembly in your solution can still invoke methods in your other assemblies, without sharing so much as a plain old data (POD) structure with the other party. You basically have a constant “yet another layer of abstraction” at your disposal. Or put another way, your components have been atomized.
Let’s compare this to what you’d have to do in traditional OOP. First, assuming you’re using a design-by-interface approach, you’d have an interface and at least one implementation of the interface. Depending on scenario, you might want to use an abstract factory scheme for creating objects. So that makes two classes and an interface, which could easily total 30 or more lines of code. In addition, if you wanted to create a true plug-in from your implementation, you’d end up with one project for the consumer parts, which consumes your plug-in, and one project for the implementation parts—the project with your class that implements the interface. And if you were to go with a completely modular approach, you might use a third project to contain your interface and possibly your abstract factory. Then, as you looked at the result, you’d realize you ended up with at least three cross-component references among your three projects. Moreover, you can’t add to or change the input or output parts of your interface without having to modify all three projects. Compare that to the one-line solution, with no references at all, that the Active Event example gives you, which doesn’t even care what input or output arguments you supply to it. The traditional approach could well have twice as many lines of code, or even more, resulting, in my experience, in a far more complex dependency graph, far more rigid result, and far more complex code.
OOP Is Not OO
I love OOP. It’s a beautiful paradigm that easily allows you to encapsulate code logic, methods and data. However, one can argue that OOP hasn’t fully solved all coding issues. In some situations when using OOP you must define multiple classes, which are too often closely coupled, and create dependencies.
In OOP, you need to know a method’s signature; you need to have access to an interface; and you need to know how to create an instance of this interface. In addition, you often need to share types between the consumer and the implementation of your interface. This creates dependencies among your building blocks. Using OOP, it isn’t uncommon to have to create multiple classes just to create a simple “Hello World” plug-in. If you change an OOP interface, unless you’ve designed your interface very carefully, you may need to recompile multiple assemblies. You can end up with much of your code being abstract interfaces and boilerplate, or just having to accept the fact that a change in an interface may have a domino effect and require significant additional code changes. Whenever something is excessively complex or requires a lot of tedious, repetitive work, your gut should tell you that it’s wrong.
Ask yourself this question: “If OOP provides the ability to create object-oriented (OO) software, why is it so difficult to use it successfully?” In my opinion, OOP isn’t equivalent to OO. If it were, you wouldn’t need to know so many design patterns to fully take advantage of it.
I believe OOP, as currently practiced, has several fundamental limitations, and all those design patterns you need to implement to adequately use OOP are but a symptom of a much more fundamental problem in its architecture: the inability to deliver OO software. Active Events fix these problems, not by implementing better abstract factories or new class constructs, but rather by changing the very way methods and functions are invoked, completely taking OOP out of the equation. To create OO software, what’s needed isn’t new ways of instantiating abstract classes and interfaces, it’s a different mechanism to invoke functionality, as well as the ability to have one common signature for every interface method that’s invoked.
Software That Almost Doesn’t Exist
To see an example of just how far you can bring Active Events, check out Phosphorus Five at github.com/polterguy/phosphorusfive. Phosphorus Five is a large software project, but the Active Events implementation is only about 1.000 lines of code. You can find the Active Events implementation in core/p5.core.
In this project I created a new programming language, Hyperlambda, for creating Active Events hooks. This allows me to change the implementation of my for-each and while statements if I want. I can also scale out my else statements, to be executed on another server. And I can easily extend the programming language to create my own domain-specific keywords, for doing whatever my domain problems require.
I also created a file format that allows me to declare a Node structure dynamically, as a piece of text, and store it to my disk or my database. Furthermore, I created my own expression syntax, allowing me to reference any nodes within any part of my tree. This results in what I like to refer to as a non-programming language, yet one that’s 100 percent Turing complete. My non-programming language is neither dynamically interpreted nor statically compiled and is sometimes able to do, with five lines of code, what requires hundreds of lines of code in other programming languages.
With Active Events, together with Hyperlambda and a managed AJAX library (containing just one AJAX widget), I’ve managed to create something that I don’t even know how to talk about, without bringing in names such as “Web OS.” There are roughly 30 projects in Phosphorus Five, and there are no references among the different plug-ins. All projects simply reference the p5.core, which is the Active Events implementation, as well as p5.exp, the expression engine. That’s it. The main Web site project, p5.website in the core folder, contains only one simple container widget and almost no logic. All plug-ins are dynamically loaded in my Global.asax during application startup. Still, all projects dynamically invoke functionality inside of other projects. No references, no dependencies, no problems!
Back to Basics
The cure is always in the problem. Some of the very same problems OOP was invented to fix—global functions and data—paradoxically become the cure for the problems OOP unintentionally created. If you take a look at the Active Events design pattern, the first thing you realize is that, to some extent, this is going back to basics, but with global functions instead of methods and classes. However, because no signatures or types need to be known and shared between the consumer and the implementation of an active event, you have more of a black box environment than you might have with OOP. This allows you to easily exchange, for instance, SaveToDatabase with InvokeWebService or SaveToFile. No interfaces, no types, no POD structures, no classes, and only one commonly shared signature. Only good old, simple data. Data in, data out!
Polymorphism is as easy as changing a string. Here’s an example of how to implement polymorphism using Active Events:
string myEvent = "some-active-event";
if (usePolymorphism) {
myEvent = "some-other-active-event";
}
context.Raise (myEvent);
I realize this polymorphism construct must appear ridiculously naive and simple to the seasoned architect. However, its simplicity is the reason it works. With Active Events, you can fetch the name of a method or function from a database, a configuration file or even from a user providing its name through a textbox in your form. You can think of this as a variation of polymorphism that doesn’t require explicit classes. This is polymorphism without types. This is polymorphism dynamically determined during runtime. By taking all traditional ideas about polymorphism out of the equation, and refactoring the very essence of polymorphism, you end up with actual working polymorphism—encapsulation and polymorphism, without classes, types, interfaces or design patterns. Chaining your Active Events becomes as easy as combining atoms into molecules. That is agile software!
Node.cs, the Last Graph Type You’ll Ever Need
With Active Events, you exclusively send data in and out from your active events. This is what lets you loosely couple your components. To do this, you need a data class—the only graph type you’ll need when using an Active Events paradigm. Your class needs to be able to wrap all possible fields and properties from all possible classes. In Phosphorus Five, this class is called Node.cs and it’s simply a graph object, with a key/value/child design.
The key to successfully implementing Active Events is that the Node class is the only argument your Active Events are allowed to take as input and return as output. It just so happens that almost all classes can be effectively reduced to a key/value/child graph POD object. This is what, combined with the dynamic loading of the Active Events assemblies, significantly reduces the number of dependencies between projects.
The Node.cs implementation needs to be able to hold a key or name, a value that can be any object, and a “children” collection of Nodes. If your Node class conforms to these constraints, you can easily transform almost all possible objects to a Node instance. For those acquainted with JSON or XML, the similarities may be obvious at this point. The following is simplified pseudo code demonstrating the Node class structure:
class Node
{
public string Name;
public object Value;
public List<Node> Children;
}
Internally, within your components, you can use as much OOP as you wish. However, when a component needs to invoke logic in another component, all the input data must somehow be transformed to a Node instance. When information is returned from Active Events, the same needs to be done. But within your components, you’re free to use any classes, interfaces, abstract factories, facade components, singletons, and design patterns you wish. Externally, though, there’s only the Node, and Active Events becomes the bridge between the components. Think of Active Events as the protocol, and Nodes as its data, if it helps you to create a mental image of the relationship.
Wrapping Up
Although intended to replace most of the other design patterns in the world today, Active Events is not a silver bullet. Among other things, this technology comes with some overhead. For instance, instead of invoking methods directly, you go through a Dictionary lookup. In addition, the methods use some reflection. These indirect method invocations are probably orders of magnitudes more expensive than traditional virtual method invocation. Moreover, you need to transform your objects to nodes and data wherever you interface with other components.
However, active events are not supposed to replace everything you do. The idea is to provide better interfacing between components, and for this, the performance overhead is not a primary concern. Whether it takes 5 CPU cycles or 500 CPU cycles to invoke your SaveToDatabase method is completely irrelevant if its actual implementation takes 5,000,000 CPU cycles! Donald Knuth once said, “Premature optimization is the root of all evil.”
Every time you think of writing an interface, you should ask yourself whether it would be better to create an active event instead. Once you’ve gotten some experience with Active Events, the answer to that question tends to become, “Probably yes.” With Active Events, interfaces and abstract classes are for the most part reduced in emphasis.
I know that sounds like an absurdly bold statement, but check out the Phosphorus Five project discussed in “Software That Almost Doesn’t Exist.” In Hyperlambda, the “language” I created for Active Events, an object can be a file, a folder, a lambda callback, a subpart of a graph Node tree, a piece of text taken from a database, or a piece of data sent over HTTP. And all objects can be executed, as if they were machine-understandable execution trees. In Hyperlambda, you could, in theory, execute the number 42.
I first thought of Active Events more than seven years ago, and intuitively felt they held an intrinsic beauty. The problem is that they challenge 60 years of conventional programming wisdom. Active Events become the point where even our best practices must be refactored. It took me seven years to unlearn what I had previously learned and to trust my intuition. The biggest problem with Active Events is actually not in their implementation, it’s in your head. The fact that I have entirely recreated Phosphorus Five five times is a testament to that fact.
In the first draft of this article, I created dozens of analogies. By hooking into your pre-existing knowledge about other subjects, such as physics and biology, I attempted to convince you about the superiority of Active Events. In my second draft, I tried to provoke you. The idea was to motivate you to prove me wrong, such that you would look for flaws in my statements, without being able to find any, of course. However, in my third and final draft, I decided to simply tell you about Active Events. If you can get your head to go along, Active Events will be the last design pattern you’ll ever have to learn.
Thomas Hansen has been creating software since he was 8 years old, when he started writing code using the Oric-1 computer in 1982. Occasionally he creates code that does more good than harm. His passions include the Web, AJAX, Agile methodologies and software architecture.
Thanks to the following Microsoft technical expert for reviewing this article: James McCaffrey