August 2015

Volume 30 Number 8

Cutting Edge - CQRS and Events: A Powerful Duo

By Dino Esposito | August 2015

As with many aspects of modern software, nothing is really new. Things are often relabeled and presented with new and compelling buzzwords. Command Query Responsibility Segregation (CQRS) is a great example. Domain events that describe observable changes in the business domain are another.

While defining the Eiffel programming language back in the late 1980s, Bertrand Meyer formalized the principle at the foundation of what is today called CQRS. In software, any basic action can be either a command or a query—but never both. If it’s a command, it’s expected to alter the state of the system. If it’s a query, it’s expected to report about the state of the system without altering it in any way.

In other words, asking a question shouldn’t change the answer. This is sometimes referred to as the Command Query Separation (CQS) principle. That’s fine for commands and queries, but what about domain events?

Events in Business Applications

Since the early days of software engineering, architects have designed line-of-business applications able to survive changes in business and track those changes. To support business intelligence and statistical analysis, architects have also managed to make sequences of actions repeatable to some extent. They didn’t call them “domain events” or coin cool terms such as “event sourcing” right away, yet the concept of domain events has been used extensively for years in business systems. This is especially true for accounting, finance and banking systems.

Then we had the Domain Driven Design (DDD) revolution and lost in this “new” universal software model, I think everyone missed some perspective on software architecture. Developers mostly focused on layers and disregarded vertical segments of systems such as command and query stacks.

In my view, the difference between the CQS principle formalized by Bertrand Meyer and today’s CQRS principle is all in where it’s applied. CQS is more of a universal principle for software development. CQRS touches on the system architecture. When you apply CQS at the architecture level, you have CQRS.

I first presented the entry-level version of CQRS in the article, “CQRS for the Common Application” (msdn.microsoft.com/magazine/mt147237). I went a bit deeper in the subsequent article, “CQRS and Message-­Based Applications” (msdn.microsoft.com/magazine/mt238399).

These articles described the command stack based on messages being exchanged between the presentation and business logic via the application layer. I briefly touched on business events you can save to an ad hoc store to log relevant business items, such as a product order or cancellation.

The focus was more on using messages to implement business workflows triggered by commands. In that context, events were just cross-handler notifications of recent occurrences to which handlers may need to react. This is the first step of a longer evolution aimed at transitioning software architects from the idea of “models-to-persist” to the idea of “events-to-log.” I see CQRS as the starting point of a change that will have a profound impact on system architecture.

The Apple of Sir Isaac Newton and I

You’ve heard the story of the apple that fell on Sir Isaac Newton’s head that led him to formulate the Universal Law of Gravity. It’s probably more legend than fact, but it’s true that many discoveries start with a curious event. I recently had my own version of an apple falling on my head that made me think seriously about CQRS and events. I was doing some preliminary analysis for rebuilding a system a customer has been using for about five years.

The customer’s system was an ASP.NET MVC 2 Web site with a plain SQL Server back-end database. The data access layer was centered on a relational database consumed via Entity Framework. Inferred entity classes were extended with additional, business-specific methods using the partial class mechanism of the Microsoft .NET Framework. Cross-entities behavior was delegated to domain services. Although I wouldn’t call it a reference implementation of the Domain Model pattern, it had bits and pieces of DDD facts here and there. Generally, I’d call it a plain Create, Read, Update, Delete (CRUD) system with some amount of business logic to sit as a buffer between the back end and the rest of the system.

In software, a CRUD system is an application built around a bunch of database tables. The layers surrounding the database validate and lay the groundwork for the core CRUD operations against the database. That was the system.

Another aspect of a CRUD system that often goes unnoticed, or simply doesn’t get enough attention, is it uses the database as a snapshot. At any time, the database behind a CRUD system stores the last known good system state. Sometimes this is enough, sometimes not.

The apple that fell on my head made me realize a snapshot database approach was OK for version one. With version two, the customer wants more. Now the customer wants to be able to track history and repeat sequences of steps under different conditions to determine the outcome. When it comes to this, the snapshot database as the sole (or primary) form of storage is no longer appropriate.

A Sample Booking Application

Consider an application that lets users book a meeting room. At some point, the presentation layer will display a screen similar to Figure 1.

Figure 1 A Mockup for the Rooms Timesheet

Imagine each room has its own business rules concerning booking. For example, users can book the White room for one hour from 8 to 10 and then from 13 onward. The Blue room has 30-minute time slots available from 9:30 in the morning. Finally, the Green room is available starting at 9 a.m. for one-hour increments. These details are saved and matched to existing bookings. You can generate a grid where you can gray out slots already taken and make clickable slots available.

The rules governing room availability aren’t set in stone and may indeed change. No big deal, you say. If it changes, let the admin edit the rules accordingly. This approach works nicely as long as you’re using the mockup of Figure 1 to present the latest system state.

The moment the customer asks to be able to navigate between the current grid and the grid three weeks before, you’re lost. More precisely, you’re lost if any changes to the booking business rules happen within the past three weeks. You know the current slots and times for each room, but you’ve lost the details of what it was three weeks before.

When the customer brought this up, I thought fixing it was simply a matter of bringing some missing information to the table. All I have to do is save some rules and read them back when needed. As simple as it may sound, properly addressing such a small piece of missing information requires a deep architectural change. It’s like the tip of the iceberg—and the iceberg is switching from a snapshot database to event-based data representation.

Event sourcing refers to event-based data representation. I won’t cover event sourcing in-depth here, but that’s precisely the topic for the next column. For the rest of this article, I’ll discuss a mixed approach that weds database snapshots and business event logs. It’s probably a good first step to transition existing classic architectures to the future. More than any fancy new technology, framework, release and runtime, the big thing of the next few years is primarily architectural—event-based data as the primary data source instead of data snapshots (as shown in Figure 2).

Figure 2 The Underwater Iceberg of Events

Beyond Data Snapshots

For decades, most of us built systems concerned only with the latest current system state. In this scenario, databases stored entity snapshots (or aggregates if you feel more comfortable with the DDD terminology) and it worked beautifully. If you’ve never felt the need to log business events as they occur, then you probably won’t.

The moment the customer requests a feature for which the application clock must be moved backward to a specific time, though, you’re lost. The only way out is to re-architect the system to save business events. Suppose now you have a table similar to the one in Figure 3.

Figure 3 A Sample Table Storing Booking Rules for Rooms

This table indicates you can book room 1 according to different rules between March 1 and June 1. It also means you must redesign the Figure 1 grid according to the date to which it refers.

The record in the RoomRules table is logically equivalent to a business event, in that it indicates an event occurred to change the booking rule. In terms of implementation, though, it’s a plain record in an additional table and requires a bunch of extra queries (one per room) before populating the grid of Figure 1.

A new booking rule just requires a new record to the table. Diligent developers, however, would probably already have some sort of rules table. They would provide one row per room. In this context, the only difference seems to be an extra column called ValidSince in Figure 3.

Coming from a classic snapshot storage vision, using the word “business event” to define a bunch of additional records in an additional table might seem like overkill. Yet this is precisely the perspective you get when you’re right on the waterline with snapshots above you and the entire world of event sourcing just beneath.

In more specific terms, the rules for booking rooms are events and logging them is tracking the history of the system. Getting the rules to apply to a given room is “replaying” the events for that room that have occurred since the system started and up to a given point. You may not have the current status saved somewhere, but you have all events logged. To get the system state, just replay events and build a valid instance of the Room entity.

Does this sound weird? This is how any banking software calculates your current “balance.” The balance never comes out of a plain read. More likely, it’s a snapshot persisted to a given date and replaying within all subsequent events.

Events and CQRS Together

Events empower software beyond imagination. Even plain CRUD systems are touched as soon as customers request simple forms of business intelligence and statistical analysis. Events are immutable. Because of that, you can easily replicate event databases to new instances of software. This is a key statement of scalability. At the same time, having command and queries separated (and thus distinct stacks for processing commands and executing queries) lets you focus on events in the command stack and have snapshot tables ready for quick reads in the query stack.

If you have performance concerns, you should save events when you run a command and update all projections of data you need for queries. For example, when you add a new booking rule, from now on that’s the default. You can update an additional current settings table you can quickly query without replaying events. You can then go back and replay events as required.

Wrapping Up

With events, you don’t miss a thing. You still have your business run and start teaching yourself a more powerful type of architecture. By storing data as events, you take one step down the data abstraction level—meaning you can use the same amount of lower-level data to build any number of projections and snapshots for many business purposes such as plain queries, business intelligence, what-if analysis, statistics, usage analytics and anything else you can imagine.

Dino Esposito is the coauthor of “Microsoft .NET: Architecting Applications for the Enterprise” (Microsoft Press, 2014) and “Programming ASP.NET MVC 5” (Microsoft Press, 2014). A technical evangelist for the Microsoft .NET Framework and Android platforms at JetBrains and frequent speaker at industry events worldwide, Esposito shares his vision of software at software2cents.wordpress.com and on Twitter at twitter.com/despos.

Thanks to the following technical expert for reviewing this article: Jon Arne Saeteras