Cutting Edge - CQRS and Message-Based Applications

By Dino Esposito | July 2015

Dino Esposito At the end of the day, Command and Query Responsibility Segregation (CQRS) is software design that separates the code that alters state and the code that just reads state. That separation can be logical and based on different layers. It can also be physical and involve distinct tiers. There’s no manifesto or trendy philosophy behind CQRS. The only driver is its simplicity of design. A simplified design in these crazy days of overwhelming business complexity is the only safe way to ensure effectiveness, optimization and success.

My last column (msdn.microsoft.com/magazine/mt147237) offered a perspective of the CQRS approach that made it suitable for any type of application. The moment you consider using a CQRS architecture with distinct command and query stacks, you start thinking of ways to separately optimize each stack.

There are no longer model constraints that make certain operations risky, impractical or perhaps just too expensive. The vision of the system becomes a lot more task-oriented. More important, it happens as a natural process. Even some domain-driven design concepts such as aggregates stop looking so irksome. Even they find their natural place in the design. This is the power of a simplified design.

If you’re now curious enough about CQRS to start searching for case studies and applications close to your business, you may find most references refer to application scenarios that use events and messages to model and implement business logic. While CQRS can happily pay the bills with far simpler applications—those you might otherwise label as plain CRUD apps—it definitely shines in situations with greater business complexity. From that, you can infer greater intricacy of business rules and high inclination to change.

Message-Based Architecture

While observing the real world, you’ll see actions in process and events that result from those actions. Actions and events carry data and sometimes generate new data, and that’s the point. It’s just data. You don’t necessarily need a full-fledged object model to support executing these actions. An object model can still be useful. As you’ll see in a moment, though, it’s just another possible option for organizing business logic.

A message-based architecture is beneficial as it greatly simplifies managing complex, intricate and frequently changing business workflows. These types of workflows include dependencies on legacy code, external services and dynamically changing rules. However, building a message-based architecture would be nearly impossible outside the context of CQRS that keeps command and query stacks neatly separated. Therefore, you can use the following architecture for the sole command stack.

A message can be either a command or an event. In code, you’d usually define a base Message class and from that, define additional base classes for commands and events, as shown in Figure 1.

Figure 1 Defining the Base Message Class

public class Message
{
  public DateTime TimeStamp { get; proteted set; }
  public string SagaId { get; protected set; }
}
public class Command : Message
{
  public string Name { get; protected set; }
}
public class Event : Message
{
  // Any properties that may help retrieving
  // and persisting events.
}

From a semantic perspective, commands and events are slightly different entities and serve different but related purposes. An event is nearly the same as in the Microsoft .NET Framework: a class that carries data and notifies you when something that has occurred. A command is an action performed against the back end that a user or some other system component requested. Events and commands follow rather standard naming conventions. Commands are imperative like SubmitOrderCommand, while events are in past tense such as OrderCreated.

Typically, clicking any interface element originates a command. Once the system receives the command, it originates a task. The task can be anything from a long-running stateful process, a single action or a stateless workflow. A common name for such a task is saga.

A task is mono-directional, proceeds from the presentation down through the middleware and likely ends up modifying the system and storage state. Commands don’t usually return data to the presentation, except perhaps some quick form of feedback such as whether the operation completed successfully or the reasons it failed.

Explicit user actions aren’t the only way to trigger commands. You can also place a command with autonomous services that asynchronously interact with the system. Think of a B2B scenario, such as shipping products, in which communication between partners occurs over an HTTP service.

Events in a Message-Based Architecture

So commands originate tasks and tasks often consist of several steps that combine to form a workflow. Often when a given step executes, a results notification should pass to other components for additional work. The chain of sub-tasks triggered by a command can be long and complex. A message-based architecture is beneficial because it lets you model workflows in terms of individual actions (triggered by commands) and events. By defining handler components for commands and subsequent events, you can model any complex business process.

More important, you can follow a working metaphor close to that of a classic flowchart. This greatly simplifies understanding the rules and streamlines communication with domain experts. Furthermore, the resulting workflow is broken into myriad smaller handlers, each performing a small step. Every step also places async commands and notifies other listeners of events.

One major benefit to this approach is the application logic is easily modifiable and extensible. All you do is write new parts and add them to the system, and you can do so with full certainty they won’t affect existing code and existing workflows. To see why this is true and how it really works, I’ll review some of the implementation details of message-based architecture, including a new infrastructural element—the bus.

Welcome to the Bus

To start out, I’ll look at a handmade bus component. The core interface of a bus is summarized here:

public interface IBus
{
  void Send<T>(T command) where T : Command;
  void RaiseEvent<T>(T theEvent) where T : Event;
  void RegisterSaga<T>() where T : Saga;
  void RegisterHandler<T>();
}

Typically, the bus is a singleton. It receives requests to execute commands and event notifications. The bus doesn’t actually do any concrete work. It just selects a registered component to process the command or handle the event. The bus holds a list of known business processes triggered by commands and events, or advanced by additional commands.

Processes that handle commands and related events are usually referred to as sagas. During initial bus configuration, you register handler and saga components. A handler is just a simpler type of saga and represents a one-off operation. When this operation is requested, it starts and ends without being chained to other events or by pushing other commands to the bus. Figure 2 presents a possible bus class implementation that holds sagas and handlers references in memory.

Figure 2 Example of a Bus Class Implementation

public class InMemoryBus : IBus
{
  private static IDictionary<Type, Type> RegisteredSagas =
    new Dictionary<Type, Type>();
  private static IList<Type> RegisteredHandlers =
    new List<Type>();
  private static IDictionary<string, Saga> RunningSagas =
    new Dictionary<string, Saga>();
  void IBus.RegisterSaga<T>() 
  {
    var sagaType = typeof(T);
    var messageType = sagaType.GetInterfaces()
      .First(i => i.Name.StartsWith(typeof(IStartWith<>).Name))
      .GenericTypeArguments
      .First();
    RegisteredSagas.Add(messageType, sagaType);
  }
  void IBus.Send<T>(T message)
  {
    SendInternal(message);
  }
  void IBus.RegisterHandler<T>()
  {
    RegisteredHandlers.Add(typeof(T));
  }
  void IBus.RaiseEvent<T>(T theEvent) 
  {
    EventStore.Save(theEvent);
    SendInternal(theEvent);
  }
  void SendInternal<T>(T message) where T : Message
  {
    // Step 1: Launch sagas that start with given message
    // Step 2: Deliver message to all already running sagas that
    // match the ID (message contains a saga ID)
    // Step 3: Deliver message to registered handlers
  }
}

When you send a command to the bus, it goes through a three-step process. First, the bus checks the list of registered sagas to see if there’s any registered sagas configured to start upon receipt of that message. If so, a new saga component is instantiated, passed the message and added to the list of running sagas. Finally, the bus checks to see if there’s any registered handler interested in the message.

An event passed to the bus is treated like a command and routed to registered listeners. If relevant to the business scenario, however, it may log an event to some event store. An event store is a plain append-only data store that tracks all events in a system. Using logged events varies quite a bit. You can log events for tracing purposes only or use that as the sole data source (event sourcing). You could even use it to track the history of a data entity while still using classic databases for saving the last-known entity state.

Writing a Saga Component

A saga is a component that declares the following information: a command or event that starts the business process associated with the saga, the list of commands the saga can handle, and the list of events in which the saga is interested. A saga class implements interfaces through which it declares the commands and events that are of interest. Interfaces like IStartWith and ICanHandle are defined as follows:

public interface IStartWith<T> where T : Message
{
  void Handle(T message);
}
public interface ICanHandle<T> where T : Message
{
  void Handle(T message);
}

Here’s an example of the signature of a sample saga class:

public class CheckoutSaga : Saga<CheckoutSagaData>,
       IStartWith<StartCheckoutCommand>,
       ICanHandle<PaymentCompletedEvent>,
       ICanHandle<PaymentDeniedEvent>,
       ICanHandle<DeliveryRequestRefusedEvent>,
       ICanHandle<DeliveryRequestApprovedEvent>
{
  ...
}

In this case, the saga represents the checkout process of an online store. The saga starts when the user clicks the checkout button and the application layer pushes the Checkout command to the bus. The saga constructor generates a unique ID, which is necessary to handle concurrent instances of the same business process. You should be able to handle multiple concurrently running checkout sagas. The ID can be a GUID, a unique value sent with the command request or even the session ID.

For a saga, handling a command or event consists of having the Handle method on the ICanHandle or IStartWith interfaces invoked from within the bus component. In the Handle method, the saga performs a calculation or data access. It then posts another command to other listening sagas or just fires an event as a notification. For example, imagine the checkout workflow is as shown in Figure 3.

Figure 3 The Checkout Workflow

The saga performs all steps up to accepting payment. At that point, it pushes an AcceptPayment command to the bus for the PaymentSaga to proceed. The PaymentSaga will run and fire a PaymentCompleted or PaymentDenied event. These events will again be handled by the CheckoutSaga. That saga will then advance to the delivery step with another command placed against another saga interacting with the external subsystem of the shipping partner company.

The concatenation of commands and events keeps the saga live until it reaches completion. In that regard, you could think of a saga as a classic workflow with starting and ending points. Another thing to note is that a saga is usually persistent. Persistence is typically handled by the bus. The sample Bus class presented here doesn’t support persistence. A commercial bus such as NServiceBus or even an open source bus like Rebus might use SQL Server. For persistence to occur, you must give a unique ID to each saga instance.

Wrapping Up

For modern applications to be truly effective, they must be able to scale with business requirements. A message-based architecture makes it incredibly easy to extend and modify business workflows and support new scenarios. You can manage extensions in total isolation, All it takes is adding a new saga or a new handler, registering it with the bus at application startup and letting it know how to handle only the messages it needs to handle. The new component will automatically be invoked only when it’s time and will work side by side with the rest of the system. It’s easy, simple and effective.

Dino Esposito is the co-author of “Microsoft .NET: Architecting Applications for the Enterprise” (Microsoft Press, 2014) and “Programming ASP.NET MVC 5” (Microsoft Press, 2014). A technical evangelist for the Microsoft .NET Framework and Android platforms at JetBrains and frequent speaker at industry events worldwide, Esposito shares his vision of software at software2cents.wordpress.com and on Twitter at twitter.com/despos.

Thanks to the following technical expert for reviewing this article: Jon Arne Saeteras