Event Sourcing pattern for microservices in .Net Core

Event Sourcing pattern is a Domain Driven Design pattern that defines an approach to handling operations on data that’s driven by a sequence of events, each of which is recorded in an append-only store.

Why should we follow an architecture pattern?

Every project’s software development life cycle has 2 opposing forces, the force of doing things and the force of doing things right.

While both lead to a working software, the force of doing things, or the force of getting things done as I like to call it, leads to a greater amount of rework.

By not following a tried and tested architecture pattern and just trying to get a tangible output asap always leads to rework. The rework may be due to the issues faced at a later point in the life cycle due to the incorrect domain definition or frequent changes in design due to the lack of domain knowledge at the design phase. Rework could also be due to the inevitable technical debt which one has to pay at some point.

What is Event Sourcing pattern? How is it different?

The typical approach for most applications that work with data is to maintain the current state of the data by updating it as users work with it. For example, in the traditional CRUD model a typical data process is to read data from the store, make some modifications to it, and update the current state of the data with the new values—often by using transactions that lock the data.

The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied in the EventStore. Not only can we query these events, we can also use the event log to reconstruct past states.

Some key facts to remember while implementing Event Sourcing pattern

1. An event is something that has happened in the past.
2. Events are expressions of the ubiquitous language.
3. Events are not imperative and are named using past tense verbs.
4. Have a persistent store of events. (Event Store)
5. Append-only, no delete. A delete or update action(updateEvent) will also be appended in the event store to maintain the history but no existing records will be modified.
6. Replay the (related)events to get the last known state of the entity. (Event aggregator)

How does Event Sourcing work?

Let’s take an example of a conference management system that needs to track the number of completed bookings for a conference so that it can check whether there are seats still available when a potential attendee tries to make a booking.

In our conference management system we have 3 microservices for the sake of this example. This example also implements the CQRS pattern along with the Event Sourcing pattern.

• Reservation : On receiving the CreateReservationCommand or UpdateReservationCommand it creates the corresponding ReservationCreatedEvent/ReservationUpdatedEvent and publishes the event to the microservices listening to the event(s) on the message bus. It also checks for the availability of seats for a given conference and availability of the attendee at the specific time slot by using GetConferenceDetailsQuery and GetAttendeeStatusQuery respectively. It also persists the event to the Event Store which is used when a GetConferenceReservationStatusQuery is received. The Event Aggregator is used to create the current reservation state of a conference by aggregating the sequence of events stored in the Event Store.
• Attendee : On receiving the ReservationCreatedEvent/ReservationUpdatedEvent in the Event Store, an attendee object is created/updated in the database. On receiving the GetAttendeeStatusQuery to check the attendee’s availability for a specific time slot, the Event Aggregator is used to aggregate the sequence of ReservationEvents and find out the availability of the attendee for the specific event.
• Conference : The Conference microservice uses the Event Aggregator to aggregate the ReservationCreatedEvent/ReservationUpdatedEvent in the Event Store to create the current state of number of seats available for the conference. This can be used to send the response for GetConferenceReservationStatusQuery.

Now let’s find out what the terms Event Store and Event Aggregator, which were repeatedly used in our example, mean.

Event Store

In software, persistence is made of four key operations through which developers manipulate the state of the system. They are CREATE, UPDATE, and DELETE, to alter the state, and QUERIES to read the state without altering. A key principle of software and especially CQRS inspired software, is that asking a question should not change the answer.

Similarly an Event Store is where the Events are persisted in the form of streams of immutable events. The event store is the permanent source of information, and so the event data should never be updated. The only way to update an entity to undo a change is to add a compensating event to the event store.

The CREATE operation in an event-based persistence scenario is not much different from classic persistence(CREATE). The event is is created with the following attributes in the Event Store.

• EventId : used to uniquely identify the event.
• Event TimeStamp : used to identify when the event was created and is also used to identify the sequence of events.
• EventName : used to recognise the type of Event
• EventData : the Event object stored in a serialised form
• Aggregator Id : used to identify the entity to which the event belongs. This is used while materialising the view for the entity

EventStore table in database:

The UPDATE operation on an entity is different from a classic update. Here you don’t override an existing record, but just add another record with similar information as create operation, unique event id, timestamp of event, event name, serialised UpdateEvent data and aggregator Id.

The DELETE operations are analogous to UPDATE operations. Here also you don’t delete any existing record, but just add another record to specify the delete event. Subsequently, we can say that the deletion is logical and consists in writing that the entity with a given ID is no longer valid and should not be considered for the purposes of the business. This can be specified by the unique event name to specify the entity is deleted.

Event Aggregator

The main aspect of event sourcing is the persistence of messages, which enables you to keep track of all changes in the state of the application. Recording individual events doesn’t give you immediate notion, however, by reading back the log of events, you can rebuild the present state of the entity. This aspect is what is commonly called the replay of events and is done by the Event Aggregator.

Event aggregation is a two-step operation. First, you grab all events stored for a given aggregate in sequence using AggregateId and TimeStamp. Second, you look through all events in some way and extract information from events and copy that information to a fresh instance of the aggregate of choice.

A key function one expects out of an event-based data store is the ability to return the full or partial stream of events. This function is necessary to rebuild the state of an aggregate out of recorded events.

In our example, let’s consider the conference with conferenceId 1000129 has the capacity of 50 seats.

1. Reservation m/s receives a CreateReservationCommand to reserve 1 seat for conference 1000129 followed by 2 seats. Therefore 2 CreateEvents are stored in the event store.
2. Reservation m/s the receives a UpdateReservationCommand to cancel the reservation for 1 seat for conference 1000129. 1 UpdateEvent is stored in the Event Store.
3. Reservation m/s receives a CreateReservationCommand to reserve 1 seat for conference 1000129. 1 CreateEvent is stored in the event store.

Reservation m/s now receives the GetConferenceReservationStatusQuery. The Command Handler calls the Event Aggregator called ReservationAggregator which gets the conferenceId 1000129 from the query and fetches all events in the EventStore where AggregatorId=1000129. It then finds 4 events from the event store (3 CreateEvents and 1 UpdateEvents). On processing these events the seat availability is calculated and the SeateAvailabilityAggregate is created.
The response is sent back by the Reservation m/s specifying 47 seats out of the 50 seats are available.

Advantages of Event Sourcing pattern

• Performance improvement in insert/update of data — Events are immutable and can be stored using an append-only operation. Therefore, we would expect to have much less(or none) deadlocks(or concurrency exceptions) happening whenever and event is stored in Event Store.
• Simplification of implementation – Events don’t directly update a data store. They’re simply recorded for handling at the appropriate time. This can simplify implementation and management. However, the domain model must still be designed to protect itself from requests that might result in an inconsistent state.
• Audit trail — since state is constructed from sequence of events it is possible to extract detailed log since the begging and up to current date. This is becomes very useful for a production incident post-mortem. The list of events can also be used to analyse application performance and detect user behaviour trends, or to obtain other useful business information.
• Projections & queries — The append-only storage of events can be used to monitor actions taken against a data store, regenerate the current state as materialised views or projections by replaying the events at any time. In addition, the requirement to use compensating events to cancel changes provides a history of changes that were reversed, which wouldn’t be the case if the model simply stored the current state.

Thus we have seen how Event Sourcing pattern can be used in microservices using .net core.

Most of the content that I have referred to get a better understanding of Event Sourcing Pattern and to write this post are as below:

1. Modern Software Architecture course by Dino Esposito on Pluralsight.
2. Microsoft Docs
3. Martin Fowler’s blog

This post has been created by me as I was learning and implementing Event Sourcing pattern in the code I develop. There may be some parts missed or not completely covered. Please feel free to reach out to me, I’d love to talk about software development patterns and learn along the way.

I hope you enjoyed reading this.

Thank you!