An Introduction to Microservices pt. 6

Table of Contents

1. What are we trying to solve?
2. The Solution: Messaging
3. Show Me the Code
4. Bibliography

On part 5, we talked about Messaging. On the last part of this series, we'll talk about how to handle situations when services try to collaborate in a synchronous way, but are not able to: Circuit Breaker.

What are we trying to solve?

As we said before, our application's services often need to collaborate. They can do this asynchronously, e.g. messaging, or they can do it synchronously. When they collaborate synchronously, the called service might be unavailable or with high latency. While the caller service waits for an answer, its resources, such as threads, will be blocked. This can lead to resource exhaustion, which would make the caller unavailable too. This can cascade on our entire application. So, how can we solve this?

The Solution: Circuit Breaker

We can introduce a Circuit Breaker on our services. Each service must implement a proxy, which will work similarly to an electrical circuit. If a call to an specific service fails too many times, the circuit breaker is activated and, for a period of time, all the following calls made to that service will fail automatically. After the period expires, the circuit breaker will allow some requests through. If none of them fail, the circuit breaker is deactivated. Otherwise, the timer will be reset.

By failing fast, we'll avoid that the caller service hangs and wastes precious resources from a call that will, most likely, fail!

Benefits

• Services can fail fast, avoiding the wasting of resources.

Drawbacks

• It's difficult to choose timeout values without creating false positives or adding excessive latency.

Show Me the Code

Just for the sake of simplicity, I'll show you how to implement the Circuit Breaker pattern only on our API Gateway. If you want to dive deeper on the pattern and implement it between two services, you can head over to the Polly project on Github. Polly is the same library Ocelot uses to implement their Circuit Breaker. Now, let's get to the code!

To implement the Circuit Breaker on our API Gateway, we first need to add a package to the project:

dotnet add package Ocelot.Provider.Polly

This will add the Polly provider to our Gateway. After the installation, open the 'ocelot.json' file. As was the case with our Load Balancer, we'll need to apply our Circuit Breaker in a route per route basis. To do this, add the following to any route you want:

{
    "Routes": [
        {
            // ...
            "QoSOptions": {
                "ExceptionsAllowedBeforeBreaking": 3,
                "DurationOfBreak": 10000,
                "TimeoutValue": 5000
            }
        },
        {
            // ...
        }
    ],
    // ...
}

Let me explain this code. "QoSOptions" stands for "Quality of Service Options", here will define our rules to activate the Circuit Breaker. The first property "ExceptionsAllowedBeforeBreaking" defines the number of times our calls can fail. The "DurationOfBreak" property defines how long the Circuit Breaker will remain active, in milliseconds. Finally, the "TimeoutValue" property defines how long a request can be left hanging without a response, also in milliseconds.

We need to do one more thing to complete the setup: edit our "Startup.cs" file. We need to insert the Polly service to our Gateway:

// Startp.cs
public void ConfigureServices(IServiceCollection services)
{
    // ...
    services.AddOcelot().AddConsul().AddPolly();
    // ...
}

That's it! We're finally done with our series. You now have a basic application that has an API Gateway with Load Balancing, uses a Service Registry with Health Checks to determine services' locations and availability dynamically, asynchronous communication with messaging, and can handle synchronous communication failure gracefully!

Let me know what you think about this series on the comments below and, if you'd like to learn more about microservices, head over to microservices.io!

Bibliography

1. Microservices Patterns: Examples with Java - Chris Richardson
2. Circuit Breaker pattern - https://microservices.io/patterns/reliability/circuit-breaker.html
3. Quality of Service - Ocelot -https://ocelot.readthedocs.io/en/latest/features/qualityofservice.html
4. Polly - https://github.com/App-vNext/Polly