Transactional Operations Across Multiple Services. A Method To The Madness.

One of the many complexities that a team has to deal with in a microservices environment is transactions. Transactions that span multiple microservices. Unlike monolithic applications, where transactions are typically managed with a single database and the @Transactional
annotation, in microservices, each service often has its own database, making distributed transactions more complex. Here’s a guide on how to handle these distributed transactions effectively in Spring Boot.

First, let's begin by agreeing on what a transaction is.

A transaction is a unit of work in a computing or database environment that is treated as a single, indivisible operation. It represents a series of actions or steps that must all succeed together or fail together, ensuring data consistency and integrity, even in cases of unexpected events (like a power outage or network failure).

In a database context, a transaction might involve several queries, such as creating, updating, or deleting records. A transaction generally follows four essential properties, known as ACID properties:

a. Atomicity- All operations within a transaction are treated as a single unit. Either all operations succeed, or none do.

b. Consistency - A transaction brings the system from one valid state to another, maintaining data validity.

c. Isolation - Transactions are executed in isolation, meaning intermediate states are not visible to other transactions.

d. Durability - Once a transaction is committed, its changes are permanent and will survive system crashes.

A Short Story

In a bustling e-commerce app, imagine a customer, Alice, placing an order for a new laptop, an accessory, and express shipping. Here’s the behind-the-scenes story of how her order flows through the system, managed by the OrderSagaOrchestrator.

In a bustling e-commerce app, imagine a customer, Alice, placing an order for a new laptop, an accessory, and express shipping. Here’s the behind-the-scenes story of how her order flows through the system, managed by the OrderSagaOrchestrator.

Alice clicks "Order Now" after entering her payment and shipping information. This action kicks off a process called a saga, a carefully orchestrated series of transactions to ensure her order is processed correctly from start to finish.

Step 1: Payment Processing
The saga orchestrator first checks with the PaymentService, initiating a call to deduct the required amount from Alice's account. The paymentService.processPayment() method is called, and Alice's payment is authorized.

Step 2: Inventory Reservation
Once payment is successful, the orchestrator moves to the InventoryService, where it reserves the specific laptop model and accessory for Alice. This reservation step is crucial so that the stock isn’t sold out or given to another customer while her order is still being processed.

Step 3: Shipping Initiation
After the inventory has been successfully reserved, the saga orchestrator reaches out to the ShippingService. Here, shippingService.initiateShipping() starts the logistics, ensuring that the items are packed and ready for shipment to Alice’s address.

Handling Failures: Compensation Logic

But in a distributed environment, things can go wrong at any step. What if the shipping initiation fails because of a logistical error, or what if the inventory can’t actually be fulfilled due to a discrepancy in stock? The orchestrator is prepared with a compensation strategy.

If an exception is thrown, the orchestrator initiates compensating transactions to roll back the entire process, so Alice isn’t charged for items she won’t receive:

3.1. Cancel Shipping - The orchestrator calls shippingService.cancelShipping(), stopping the shipment.

3.2. Release Inventory - It then triggers inventoryService.releaseInventory(), freeing up Alice’s reserved items so other customers can buy them.

3.3. Refund Payment - Finally, it calls paymentService.refund() to refund Alice’s payment, ensuring she isn’t charged for the order.

In the end, this orchestrated saga ensures that Alice’s experience is smooth and consistent, and if any issues arise, they’re resolved in a way that maintains the integrity of the system. This is the magic of distributed transactions and compensation logic in microservices.

So, now that we know what a transaction is and understand a real-life scenario where this might be useful, let's dive into how to make this work in a distributed environment.

There are Key Approaches that teams utilize to solve this problem

1. SAGA Pattern: One of the most widely used patterns to handle distributed transactions in a microservices architecture is the Saga pattern. A saga is a sequence of local transactions that each service executes independently. Each step in a saga is compensated by an action that undoes it if the saga fails.

The Saga pattern can be implemented in two main ways:

1. a. Choreography-based SAGA: Each service involved in the transaction listens for events and performs its transaction. Upon completion, it emits an event that triggers the next step in the saga. If a step fails, a compensating event is triggered to undo previous steps.

2. b. Orchestration-based SAGA: A centralized service (the saga orchestrator) coordinates the saga's steps. It determines the order of operations and manages compensations if a failure occurs.

2. Two-Phase Commit (2PC): Although commonly used in monolithic systems, the two-phase commit protocol can be used across distributed systems with a distributed transaction manager like Atomikos or Bitronix. However, I would not recommend this approach because it has some limitations within the microservices context as it leads to high latency and is less fault-tolerant. I would generally avoid this approach in favor of the Saga pattern if I were you.

3. Event-Driven Architecture: Using an event-driven approach, where services communicate through events, is particularly suitable for handling distributed transactions. This approach aligns well with the Saga pattern. Each service performs its transaction independently and then emits an event to notify other services about the outcome. These events can be handled using Apache Kafka, RabbitMQ, or other message brokers.

Now, Let's see how this works in code.

There are several flavors of the saga pattern but in this article, I'll attempt to implement an orchestration-based Saga pattern in Spring Boot:

STEP 1: Defining a Saga Orchestrator:
Here, I'll create a simple service to act as the orchestrator, responsible for coordinating the transactions.

This service will define the flow of the saga, calling each service in the correct order and handling compensating transactions if needed.

@Service
public class OrderSagaOrchestrator {

    @Autowired
    private PaymentService paymentService;

    @Autowired
    private InventoryService inventoryService;

    @Autowired
    private ShippingService shippingService;

    public void createOrderSaga(Order order) {
        try {
            paymentService.processPayment(order.getPaymentDetails());
            inventoryService.reserveInventory(order.getItems());
            shippingService.initiateShipping(order.getShippingDetails());
        } catch (Exception e) {
            // Compensation logic
            shippingService.cancelShipping(order.getShippingId());
            inventoryService.releaseInventory(order.getItems());
            paymentService.refund(order.getPaymentId());
        }
    }
}

STEP 2: Creating Local Transactions and Compensation Methods in Each Service:

Each service should have its transaction for completing its step in the saga and another for compensating it if needed. Here's the general structure of how that might look like.

@Service
public class PaymentService {

    @Transactional
    public void processPayment(PaymentDetails details) {
        // Perform payment logic
    }

    @Transactional
    public void refund(String paymentId) {
        // Perform refund logic
    }
}

STEP 3: Event-Based Communication (Optional, for choreography): Each service can emit events to notify others of the transaction's outcome.

public class PaymentService {

    private final ApplicationEventPublisher eventPublisher;

    public PaymentService(ApplicationEventPublisher eventPublisher) {
        this.eventPublisher = eventPublisher;
    }

    public void processPayment(PaymentDetails details) {
        // Process payment
        eventPublisher.publishEvent(new PaymentProcessedEvent(this, details));
    }
}

STEP 4: Take steps to guarantee Data Consistency: Use idempotency checks to ensure that each step in the saga is executed only once. This is important in distributed systems, where network failures or retries can lead to duplicate requests.

STEP 5: Use a Message Broker for Reliability: If you’re using events to manage the saga, a message broker like Kafka of RabbitMq provides durability and can buffer events if a service is temporarily unavailable.

STEP 6: Error Handling and Retries: Incorporate error handling and retry logic in your orchestrator and individual services to handle temporary failures. Spring Retry is useful here, as it can automatically retry failed operations within a configurable policy.

@Retryable(value = {RemoteServiceException.class}, maxAttempts = 3, backoff = @Backoff(delay = 2000))
public void reserveInventory(List<Item> items) {
    // Attempt to reserve inventory
}

Conclusion

Distributed transactions in microservices is challenging, but by using patterns like Saga (especially with orchestration) and event-driven communication, you can achieve reliable and scalable solutions.

Spring Boot makes this easier by offering support for transactional management, event publication, and integration with message brokers.

In the end, this orchestrated saga ensures that Alice’s experience is smooth and consistent, and if any issues arise, they’re resolved in a way that maintains the integrity of the system. This is the magic of distributed transactions and compensation logic in microservices.