Event Sourcing

The journey to break down the monolith to microservices in order to accelerate software delivery lies many challenges, including defining bounded contexts as DDD aggregates; connecting the contexts with domain events using either orchestration or choreography; and the mechanism to publish these domain events reliably and persist and consume the events consistently.

An event-centric way of structuring the business logic and aggregate persistence is called event sourcing.

Problem with Traditional Persistence

Traditionally, aggregates are represented as classes or objects in applications. It has worked well for monolith systems. Despite being hard to change, maintain and scale, there are also several limitations to this approach.

Object-Relational Impedance Mismatch

The object-relational impedance mismatch problem describes the conceptual mismatch in serving relational data using an object-oriented programming language. This tends to have a few common symptoms.

• Relationships are represented in the connections between several tables in relational databases; while it is represented as object references in OOP.

• These connections are flat in nature in relational databases; but lead to hierarchical data structures in OOP.

• Whereas encapsulation is a defining feature of OOP, there is no such concept in using relational databases.

Loss of Aggregate History

The traditional persistence will only record the current state of an aggregate. Any update or delete operation will result in permanent loss of its previous state or data. This could prove to be challenging for regulatory reporting: the system needs to implement a separate business logic to provide aggregate history, and ensure the data is in sync ultimately.

Misalignment in Audit Logging

Financial systems require maintaining an audit log to keep track of the users or systems that initiated changes. This could be for security or regulatory reasons, or it could be a functional requirement to display the change history. Auditing is considered critical, yet often implemented secondary to the core features, and it can be implemented in different ways, causing the misalignment in the audit trail.

Coupled Event Publishing

As Martin Fowler put it:

Domain events capture the memory of something interesting which affects the domain. 

It is the essence of designing and architecting event-driven interactions. It should be the minimal unit of information published by an aggregate to inform its state changes.

The problem with the conventional model is: the event generation and publishing are coupled with the business logic, and it has to be synchronous with the logic itself to maintain the integrity and correctness of the events. But as described in the two-phase commit, this objective is virtually impossible to achieve. 

So how do we solve the coupling problem, as well as the issues mentioned at the same time? The answer is event sourcing.

What is Event Sourcing

Event sourcing structures business logic and aggregates persistence in a way that:

• An aggregate is stored in the database as a sequence of domain events;

• Each domain event is a state change of the aggregate;

• An application recreates the current state of an aggregate by rehydrating the events;

• The business logic is structured around producing and consuming these events.

Figure 1. Event Sourcing Flow

Beyond the obvious, there are a number of considerations for event sourcing to be more production-ready.

Concurrent Event Sourcing

In the circumstances two or more requests are updating the same aggregate simultaneously, the solution to race condition issue is by introducing optimistic locking.

In short, the aggregate should maintain a version number. If a request attempts to commit the transaction, it needs to match on the version number before making the commit. The operation will be successful only if the version remains unchanged since last read. As part of a successful commit, the version number should be increased at the same time.

UPDATE TRADE_TABLE

SET VERSION = VERSION + 1 ……

WHERE VERSION = <original version>

Event Publishing with Outbox

Event sourcing is not only about persisting domain events in one system, but also responsible for reliably publishing these events to its consumers. Since event persisting is an atomic operation, a decoupled event publishing operation is very important. It should also guarantee “at-least-once” delivery to ensure the success of event sourcing architecture. 

As described quite elaborately with the two-phase commit problem, Outbox Pattern is the solution to publish events reliably. It is also worth pointing out: outbox pattern can work with simply polling events out from the outbox table; or a more sophisticated solution using transaction log tailing, if the event store of choice supports it.

Idempotency in Event Sourcing

As the events published need to guarantee their delivery by potentially sending them multiple times, it is only logical to consider securing the idempotency of the events. Idempotency Pattern has covered large grounds on the key characteristics, and implementation recommendations on both the producer and consumer ends of an event-driven architecture. 

However, there are differences in implementing relational-based and NoSQL-based event stores. The idempotent ID can simply be committed to a processed message table as part of the atomic operation in relational-based implementation; it is not quite as simple in a NoSQL implementation, the alternative is to always include idempotent IDs in the events while processing them to avoid reprocessing of the same event, and emit pseudo events with only the idempotent ID if an operation on the aggregate is a pure side effect.

Snapshots for Improved Performance

It is obvious that loading and replaying the events could be an expensive and time consuming operation as the number of events increases.

Snapshot is a solution to improve the performance issue brought about by the ever growing number of events. Rather than rehydrating events from the very beginning, the current state of the aggregate can be recreated from replaying only the events after the latest snapshot.  This can also leverage the full potential of end-of-day snapshots generated already. 

Figure 2. Event Sourcing with Snapshots

The snapshots can be prepared easily using JSON serialisation, but for more complex aggregates that need to support undo operation, memento pattern is the answer.

Event Evolution

With the rapid development of applications, it is hard to imagine the event schema will stay the same throughout eternity. While schema migration is expensive and error prone, there are certain changes that can be considered backward compatible, such as adding a new field; whilst others not, like removing a field, changing the type or the name of an existing field. 

The principle to bear in mind when designing and planning the changes is: entities should be open for extension, but closed for modification; it is also famously known as open-closed principle in SOLID.

When a non-backward-compatible change is inevitable, instead of resorting to a massive data migration effort, the application can leverage a technique known as, upcasting, where the application needs to be to recognise multiple versions of the events, but only deal with the current schema, by transforming event schema from legacy to current when an event is loaded.

Advantages of Event Sourcing

There are many benefits with event sourcing, here highlights a couple.

Reliable Event Publishing

The foundation of event-driven microservices architecture (Building Event-Driven Microservices, p79) and one major benefit of event sourcing is to allow for reliable domain event publishing when there is a state change in the aggregate.

It guarantees accurate audit logs, and the event stream can be used as a source of truth for notifications, monitoring and analytical purposes.

Durable Aggregate History

Another major benefit of event sourcing is that it stores aggregate state change history in its entirety. This is not only to provide the current state of the aggregate, but also can serve the state of the aggregate up to any point in time. This ability unlocks a number of possibilities, such as trace back, correction and alteration, etc.

Trade-offs of Event Sourcing

In spite of the overwhelming benefits of event sourcing pattern in distributed systems, there are trade-offs as well.

Event Idempotency

It should be obvious by now that: for event sourcing to function correctly, the applications need to support idempotency, as well as being able to produce idempotent events.

Event Evolution

The schema evolution remains one the hardest problems to solve. It is difficult to get right, because the need for schema migration remains real. It is also a risky one, with the number of versions growing, the code to deal with different versions of the schema can easily go out of control as well. Hence the recommendation of upcasting previously.

Event Retention

Deletion in event sourcing is a superficial operation, which marks the aggregate with an “deleted” event. Even if cost and space are not an obstacle, maintaining an inventory of aggregate history is unattainable and does not comply with the Right to be Forgotten. The solution really depends on the scenario, one possibility is to anonymise the events after a set period of time, if the events need to be persisted in its entirety.

Event Querying

No more simple select queries, like “... where trade_value=100”, because the current state of the aggregate needs to be derived for any condition to make sense at all. That is not to mention the added complexity in the case of NoSQL: to retrieve data from various places without the popular “left join” can be proven difficult, or impossible at times. A potential solution involves using the CQRS pattern.

Development Paradigm Change

The biggest challenge yet, in the face of the move to an event sourcing model, is the mindset and skill set required of the teams, from infrastructure to development and product.