API Gateway and Microservices using Kong and dotnet core in docker

Hi Friends,

If you are reading this, probably you know what an API gateway is and why you need it. If not, below is the shot version as per whatis.techtarget.com

An API gateway is programming that sits in front of an application programming interface (API) and acts as a single point of entry for a defined group of microservices. Because a gateway handles protocol translations, this type of front-end programming is especially useful when clients built with microservices make use of multiple, disparate APIs.

A major benefit of using API gateways is that they allow developers to encapsulate the internal structure of an application in multiple ways, depending upon use case. This is because, in addition to accommodating direct requests, gateways can be used to invoke multiple back-end services and aggregate the results.

In this article, we will

1. Create microservies in dotnet core and host them in a docker container.

2. Create docker compose file for our microservices

3. Setup *Kong * in docker.

4. Expose services through Kong

Lets start.

This is the end architecture that we will be achieving after this exercise.

We have 2 microservices, one for DC and one for Marvel. We will be hosting those on our internal servers. We will use Kong API Gateway to expose these services to the outside word which will be categories into Web traffic and App traffic. These calls will come to our services through different routes, we and app.

Below will be my project folder structure.

As you can see here, I have 2 folders for 2 microservices that I will be hosting in docker using docker-compse.yaml file.

Step 1 - Create Microservices

DC Service

Lets create first microservice, DC. First navigate to the folder "DC" in command prompt and use below command to create an API project.

dotnet new webapi

This should create a basic web api project with WeatherForecast controller.

Add a new controller JusticeLeague.cs for our API,

using Microsoft.AspNetCore.Mvc;

namespace DC.Controllers
{
    [ApiController]
    [Route("[controller]/[action]")]
    public class JusticeLeague : ControllerBase
    {
        [HttpGet]
        public string Strongest(){
            return "Batman!";
        }
    }
}

This is simple HTTPGet API which returns string.

Add dockerfile which will pull down necessary images, build the project in release mode and start it at port 7001.

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env

WORKDIR /app

COPY ./ ./
RUN dotnet restore DC.csproj

COPY . ./
RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build-env /app/out .
ENV ASPNETCORE_URLS=http://+:7001  
ENTRYPOINT ["dotnet", "DC.dll"]

Now lets see if its working fine.

Run below command to build the docker image for DC service and tag it with name DC

docker build -t DC .

Lets test it by running it in a container with name dc

docker run -p 7001:7001 --name=dc dc

Here I am attaching port 7001 on local machine with port 7001 inside the docker container which was specified in our docker file.

And we can see the swagger page with our API.

Marvel Service

Now navigate to the folder "Marvel" in command prompt and run below command to create the API project.

dotnet new webapi

Again, this should create a basic web api project with WeatherForecast controller.

Add a new controller Avengers.cs for our API

using Microsoft.AspNetCore.Mvc;
namespace Marvel.Controllers
{
    [ApiController]
    [Route("[controller]/[action]")]
    public class Avengers: ControllerBase
    {
        [HttpGet]
        public string Strongest(){
            return "Iron Man!";
        }
    }
}

This is a simple API that returns strongest avenger.

Now, like we did before, add docker file which will pull down necessary images, build the project in release mode and start it at port 7002.

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env

WORKDIR /app

COPY ./ ./
RUN dotnet restore Marvel.csproj

COPY . ./
RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build-env /app/out .
ENV ASPNETCORE_URLS=http://+:7002  
ENTRYPOINT ["dotnet", "Marvel.dll"]

Now lets see if its working fine.

Run below command to build the docker image for DC service and tag it with name DC

docker build -t marvel.

Lets test it by running it in a container with name marvel

docker run -p 7002:7002 --name=marvel marvel

Here I am attaching port 7001 on local machine with port 7001 inside the docker container which was specified in our docker file.

And we can see the swagger page with our API.

Alright!

At this point, we have tested our microservices in docker containers with separate docker files.

Step 2 - Host services in Docker with Docker Compose

Now, we will create a docker compose file to get rid of these manual invocations.

Lets create docker-compose.yaml file at the root level and add below content.

version: '3.7'
services: 
    dcservice:
        container_name: DC
        build: 
            context: ./DC
            dockerfile: dockerfile
        ports: 
          - "7001:7001"
    marvelservice:
        container_name: Marvel
        build: 
            context: ./Marvel
            dockerfile: dockerfile
        ports: 
          - "7002:7002"
networks: 
  default:
    name: kong-comic-net

Here we are creating 2 services, DC and Marvel. We are using context feature of the docker file so that we can use the docker files we have created and reside in the respective folders and assign the same ports at the runtime

One more important thing is we are creating a new network named kong-comic-net and attaching both of these services to it. In later part of this article, I will use the same network for KONG.

Now lets test this setup again.

docker-compose build

This should build the images for both of these services using their respective dockerfiles.

And then

docker-compose up

This should use those images, create container and run those container by exposing port 7001 and 7002.

Step 3 - Setup KONG in Docker

Kong Gateway (OSS) is an open-source, lightweight API gateway optimized for microservices, delivering unparalleled latency, performance, and scalability.

Kong Gateway listens for traffic on its configured proxy port(s) 8000 and 8443, by default. It evaluates incoming client API requests and routes them to the appropriate backend APIs. While routing requests and providing responses, policies can be applied via plugins as necessary.

Lets start with KONG setup.

Execute below commands in this order to setup KONG with PostgreSQL DB in docker.

I got these from the official documentation.

Start your database

docker run -d --name kong-database --network=kong-comic-net -p 5432:5432 -e "POSTGRES_USER=kong" -e "POSTGRES_DB=kong"  -e "POSTGRES_PASSWORD=kong" postgres:9.6

Prepare your database

docker run --rm --network=kong-comic-net  -e "KONG_DATABASE=postgres" -e "KONG_PG_HOST=kong-database" -e "KONG_PG_USER=kong" -e "KONG_PG_PASSWORD=kong"  kong:latest kong migrations bootstrap

Start Kong

docker run -d --name kong --network=kong-net -e "KONG_DATABASE=postgres" -e "KONG_PG_HOST=kong-database" -e "KONG_PG_USER=kong" -e "KONG_PG_PASSWORD=kong" -e "KONG_PROXY_ACCESS_LOG=/dev/stdout"  -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout"      -e "KONG_PROXY_ERROR_LOG=/dev/stderr"      -e "KONG_ADMIN_ERROR_LOG=/dev/stderr"      -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl"      -p 8000:8000      -p 8443:8443     -p 127.0.0.1:8001:8001     -p 127.0.0.1:8444:8444      kong:latest

Verify its running.

curl -i http://localhost:8001/

This should return 200 OK response. That means Kong is up and running and ready for configuration.

Step 4 - Expose services through Kong Gateway

Lets start by adding "services" for our APIs.

Service and Route objects let you expose your services to clients with Kong Gateway. When configuring access to your API, you’ll start by specifying a Service. In Kong Gateway, a Service is an entity representing an external upstream API or microservice — for example, a data transformation microservice, a billing API, and so on.

The main attribute of a Service is its URL, where the service listens for requests. You can specify the URL with a single string, or by specifying its protocol, host, port, and path individually.

Lets add service for our API "DC".

curl -i -X POST http://localhost:8001/services --data name=DC --data url="http://DC:7001/"

This should give "201 created" response.

Before you can start making requests against the Service, you will need to add a Route to it. Routes determine how (and if) requests are sent to their Services after they reach Kong Gateway. A single Service can have many Routes.

Lets add 2 routes our service, one will be for our browser based web app (DC-Web) and other for our mobile app (DC-App)

curl -i -X POST http://localhost:8001/services/DC/routes   --data "paths[]=/DC-Web"   --data name=DC-Web

and ..

curl -i -X POST http://localhost:8001/services/DC/routes   --data "paths[]=/DC-App"   --data name=DC-App

Now that we have routes, we can access our APIs using these routes on PORT 8000,

and

On the similar lines, we can create a service for our "Marvel" service

curl -i -X POST http://localhost:8001/services --data name=Marvel --data url="http://Marvel:7002/"

and add routes for web and App.

curl -i -X POST http://localhost:8001/services/Marvel/routes   --data "paths[]=/Marvel-Web"   --data name=Marvel-Web

curl -i -X POST http://localhost:8001/services/Marvel/routes   --data "paths[]=/Marvel-App"   --data name=Marvel-App

Now we can control and manage traffic coming from different routes individually.

In the next part, we will look at few plugins for Kong.

Thanks for reading. Please do share your inputs.