In this blog, we are covering the following topics:
- Establish a system-design team
- Define your architecture
- Using an API gateway
- Service discovery
- Inter-Service communication
- Implement data storage
One of the most important aspects of building microservices is designing the architecture. This involves deciding on the granularity of the services, defining their APIs, and implementing the services themselves. We will explore each of these steps in detail and provide examples of how they can be accomplished using Go.
Establish a system-design team
Creating a microservices system can involve many people and components, which can make it complicated. The final software is the result of everyone's input and decisions. However, making everything work together smoothly can be difficult. That's why it's crucial to have a team responsible for guiding the system's direction and behavior. This team is referred to as the system design team in this model. In this model, the system design team has three core responsibilities:
- Design Team Structure
- Establish Standards & Incentives
- Continually Improve the System
Define your architecture
To design and build microservices, engineers or architects need to determine the boundaries and responsibilities of each microservice. Identify what communication protocols & APIs are going to be used within microservices or what data storage either open-source or commercial database solutions for each service. In the previous section, we learned that the Golang community has developed numerous frameworks and tools for microservices.
However, it is essential to ensure that the chosen library or framework addresses the current problem and does not create additional challenges in the future. Decide whether you want to use a web framework like Gin, Echo, or Buffalo, or if you prefer to build your microservices from scratch keep in mind frameworks can provide useful tools for handling HTTP requests, routing, and middleware, but they may add some overhead.
Using an API gateway
Most of the time when working with microservices, you need to decide how the application client will interact with microservices. With the monolithic application, there is just one set of endpoints that can be replicated with load balancing that distributes traffic among them. In a microservices architecture, however, each microservice exposes a set of what are typically fine-grained endpoints.
An API Gateway is an implementation that acts as a single entry point for your microservices. The API Gateway handles requests from clients and passes them to your backend services while sending the responses back to the client. It's a middle-level server that sits between the client/internet and your backend microservices. The API Gateway encapsulates the internal system architecture and provides an API that is tailored to each client. API Gateway can have many responsibilities such as authentication, authorization, security, monitoring, load-balancing, rate limiting, caching, request management, and static response handling. Below is the visualization of the API gateway in microservices.
The Golang community has provided many libraries and packages that help engineers write API Gateways without writing them from scratch. Some of them are part of complete microservices frameworks while some are introduced as standalone API Gateway services. Below are some of the API Gateway tools provided by Go Community.
- Lura - Ultra-performant API Gateway framework with middlewares.
- resgate - Realtime API Gateway for building REST, real-time, and RPC APIs, where all clients are synchronized seamlessly.
- Janus - A lightweight API Gateway and Management Platform that enables you to control who accesses your API.
- Echo middleware - Echo doesn't provide an API gateway specifically but its middleware does some of the work that an API Gateway intends to do.
The API Gateway is responsible for request routing and protocol translation. It provides each of the application’s clients with a custom API. The API Gateway can also mask failures in the backend services by returning cached or default data.
Service discovery
In a traditional monolith application, you could probably hardwire the locations, but in a modern micro-services architecture, finding the need for locations is a non-trivial problem. API Gateway needs to know the location (IP and Port) of each microservice to which it communicates. Application services have dynamically assigned locations because of autoscaling and upgrades.
Service Discovery in microservices is a way of locating other services on a network. Service discovery implementations include a central server and clients that connect to the central server. Microservices applications typically run on virtualized or containerized environments, and the instances can change their locations (IP and PORT) dynamically. So there should be a mechanism that determines which microservice to invoke when a request arrives and that's where Service Discovery comes into play. Service Discovery acts as a registry where all addresses of microservices are tracked and these instances have dynamically assigned network paths. Consequently, the API Gateway like any other service client in the system, needs to use the system's service discovery mechanism either server-side discovery or client-side discovery.
Client-Side discovery - In this discovery pattern, the client is responsible for figuring out the network locations on available instances and load-balancing requests across them. It queries the service registry (a database of available instances) and then uses a load-balancing algorithm to select one of the available service instances to request. Types of Client-Side Service discoveries are Netflix Eureka, Zookeeper, and Consul.
Server-Side discovery - In this discovery pattern, a dedicated load-balancer does all the job of load-balancing. Clients make requests via a router, which queries the service registry and forwards the request to an available instance. In fact, there's no need to write discovery logic separately for each language and framework that the Service Consumer uses. Types of Server-Side Service discoveries are NGNIX and AWS ELB
The Golang community provided tools for implementing service discovery when writing microservices. Here's how you can implement service discovery in Go:
Use Service-discovery tools - To simplify finding and managing services with Go, try using a service discovery platform like Consul, etcd, or Go-Kit. These tools ensure efficient and seamless operation of your Go apps in a distributed architecture.
Register services - When a Go service starts, it should register itself with the service discovery tool by providing its name, IP address, port, and any other relevant metadata. This registration typically happens during the service's initialization phase.
Discover services - When a Go service needs to communicate with another service, it queries the service discovery tool to obtain the location (IP address and port) of the target service and the service discovery tool returns this information to the requesting service.
Handle failures and retries - In your Go service, write logic to handle failures and retry mechanisms to handle cases where service discovery fails or returns errors.
Monitor & maintain - Implement health checks to ensure the reliability of services. Continuously monitor the performance of your service discovery solution.
Inter-Service communication
One of the biggest challenges when working with microservices are communication mechanism. Microservices are distributed by nature and they require service communication on the network level. Every microservice has an instance and process, therefore services interact using inter-service protocols such as HTTP, gRPC, and AMQP (message brokers).
Microservices has a couple of communication styles has determine the direction of interaction.
Synchronous - In synchronous communication, the client expects some response after sending a request from the server that might even be blocked while it waits. HTTP or gRPC are the communication protocols that follow the synchronous communication pattern.
Asynchronous - In asynchronous communication, the client sends a request but doesn't wait for a response it doesn't block while waiting for a response, and the response itself may not be sent immediately. AMQP (Advanced Message Queuing Protocol) is a popular communication protocol that follows an asynchronous communication pattern using a publisher/subscriber model.
Implement data storage
Data Storage is an integral part of storing your data permanently. There are several implementation strategies for data storage when designing microservices. The best strategy depends on a number of factors, such as the size and complexity of your application, the type of data you need to store, and your budget. Designing a database is one of the most challenging concerns for your microservices. There are two implementation strategies when designing the data storage solution.
Within microservices architectures, there are several options for implementing data storage patterns. To provide a better understanding, here are some examples of such patterns:
Data partitioning: This pattern divides the data into multiple partitions, each of which is stored in a separate database. This can be used to improve scalability and performance, as each microservice can access only the data that it needs.
Data replication: This pattern replicates the data across multiple databases. This can be used to improve availability and fault tolerance, as the data will still be available even if one of the databases fails.
Event sourcing: This pattern stores the history of all changes to the data. This can be used to reconstruct the data state at any point in time, and it can also be used to implement complex business logic.
Golang community provides a lot of databases, relational & non-relational database drivers, cache storage, query builders, and schema migration tools. Golang supports integration with cloud-native data stores such as Amazon RDS, AWS DynamoDB, Azure Cosmos DB, etc. Below are some community-driven data storage that can be utilized in Go microservices.
- mongo-go-driver - Official MongoDB driver for the Go language.
- redigo - Redigo is a Go client for the Redis database.
- go-elasticsearch - Official Elasticsearch client for Go.
- pq - Pure Go Postgres driver for SQL database.
- dynago - Simplify working with AWS DynamoDB
If you like my article, please like and share feedback and let me know in the comments. And don't forget to follow me on Linkedin, Github, Twitter.
Peace ✌🏻
Top comments (0)