DEV Community

zhuyasen
zhuyasen

Posted on

Easily build a simple and reliable ordering system in an hour using go efficiency tools

#go #grpc #distributedsystems #gin

Order System Introduction

The order system is the core system of a trading platform, involving various complex tasks that require careful consideration of factors such as business requirements, performance, scalability, and security.

Breaking down the order system into services like order service, stock service, coupon service, and pay service, each with its independent database, is a common practice. The order processing inevitably involves distributed transactions, such as ensuring atomicity in actions like creating an order and deducting stock. In a distributed system, ensuring the atomicity of these operations faces challenges such as process crashes, idempotence issues, rollback problems, and precise compensation problems.

In a monolithic order system, using the built-in transaction support of the database can easily solve these problems. However, when services are modularized, the issues of a distributed system must be considered. Common solutions for distributed transactions include message queue-based and state machine-based approaches, both of which are relatively heavyweight, making the order system more complex. DTM, as an alternative solution for distributed transactions, significantly simplifies the architecture of the order system and elegantly addresses data consistency issues in distributed transactions.

Image description

When a frontend requests the gRPC gateway service order_gw to submit an order API, the server performs the following operations:

  • order service: Creates an order in the order table with a unique key as the order ID.
  • stock service: Deducts stock from the stock table. If the stock is insufficient, the global transaction automatically rolls back.
  • coupon service: Marks the coupon as used in the coupon table. If the coupon is invalid, the global transaction automatically rolls back.
  • pay service: Creates a payment order in the pay table and informs the user to proceed to the payment page.

Setting Up

  1. Prepare a MySQL service by quickly launching it using the script docker-compose.yaml.
  2. Import the prepared SQL into MySQL.
  1. Prepare proto files:
  1. Install the tool sponge. After installing sponge, run the following command to open the UI interface for generating code: 
sponge run
Enter fullscreen mode Exit fullscreen mode
  1. Start the DTM service using the docker-compose.yml script: 
version: '3'
services:
  dtm:
    image: yedf/dtm
    container_name: dtm
    restart: always
    environment:
      STORE_DRIVER: mysql
      STORE_HOST: '192.168.3.37'
      STORE_USER: root
      STORE_PASSWORD: '123456'
      STORE_PORT: 3306
    #volumes:
    #  - /etc/localtime:/etc/localtime:ro
    #  - /etc/timezone:/etc/timezone:ro
    ports:
      - '36789:36789'
      - '36790:36790'
Enter fullscreen mode Exit fullscreen mode

Modify the STORE_xxx related parameters and then start the DTM service:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Quickly Create Microservices for the Order System

Generate gRPC service codes for Order, Stock, Coupon, Pay, and Order_gw

Go to the sponge UI interface, click on the left menu bar Protobuf -> create microservice project, fill in the parameters, and generate service codes for order, stock, coupon, and pay services.

Quickly create the order service, as shown in the image below:

Image description

Quickly create the stock service, as shown in the image below:

Image description

Quickly create the coupon service, as shown in the image below:

Image description

Quickly create the pay service, as shown in the image below:

Image description

Quickly create the gRPC gateway service order_gw. Click on the left menu bar Protobuf -> create grpc gateway project, fill in the parameters, and click the download code button, as shown in the image below:

Image description

Rename the generated services to order, stock, coupon, pay, and order_gw. Open five terminals, each corresponding to one service.

Configure and Run the Stock Service

Switch to the stock service directory and perform the following steps:

  1. Generate and automatically merge API-related code: 
make proto
Enter fullscreen mode Exit fullscreen mode
  1. Add MySQL connection code: 
make patch TYPE=mysql-init
Enter fullscreen mode Exit fullscreen mode
  1. Open the configuration file configs/stock.yml and modify the MySQL address and account information. Change the default gRPC server port to avoid port conflicts. 
mysql:
  dsn: "root:123456@(192.168.3.37:3306)/eshop_stock?parseTime=true&loc=Local&charset=utf8mb4"

grpc:
  port: 28282
  httpPort: 28283
Enter fullscreen mode Exit fullscreen mode

  1. Add business logic code for deducting and compensating stock in the generated template code. View the code internal/service/stock.go.

  2. Compile and start the stock service:

make run
Enter fullscreen mode Exit fullscreen mode

The stock service source code is completed following the above steps.

Configure and Run the Coupon Service

Switch to the coupon service directory, and the steps are the same as configuring and running the stock service, except for the business logic code. This is the coupon service source code.

Configure and fill in the specific business logic code, then compile and start the coupon service:

make run
Enter fullscreen mode Exit fullscreen mode

Configure and Run the Pay Service

Switch to the pay service directory, and the steps are the same as configuring and running the stock service. This is the pay service source code.

Configure and fill in the specific business logic code, then compile and start the pay service:

make run
Enter fullscreen mode Exit fullscreen mode

Configure and Run the Order Service

Switch to the order service directory, and the steps are the same as configuring and running the stock service. This is the order service source code.

As order submission requires informing the DTM service of the gRPC service addresses for the order, stock, coupon, and pay services, to manage distributed transactions, the addresses need to be configured. Open the configuration file configs/order.yml and add the service addresses and DMT service address configuration, as shown below:

grpcClient:
  - name: "order"
    host: "127.0.0.1"
    port: 8282
  - name: "coupon"
    host: "127.0.0.1"
    port: 18282
    registryDiscoveryType: ""
    enableLoadBalance: false
  - name: "stock"
    host: "127.0.0.1"
    port: 28282
    registryDiscoveryType: ""
    enableLoadBalance: false
  - name: "pay"
    host: "127.0.0.1"
    port: 38282
    registryDiscoveryType: ""
    enableLoadBalance: false

dtm:
  addr: "127.0.0.1:36790"
Enter fullscreen mode Exit fullscreen mode

Add the new fields to the configuration file and update the corresponding Go code:

make update-config
Enter fullscreen mode Exit fullscreen mode

Add the business logic code for submitting orders, creating orders, and canceling orders to the generated template code. View the code internal/service/order.go.

Configure and fill in the specific business logic code, then compile and start the order service:

make run
Enter fullscreen mode Exit fullscreen mode

Configure and Run the gRPC Gateway Service order_gw

  1. Generate gRPC service connection code.

As the gRPC gateway service order_gw as the entry point for requests and needs to connect to the order service, generate the connection code. In the sponge UI interface, click on the left menu bar Public -> generate grpc connection code, fill in the parameters, and download the code.

Image description

Extract the code, move the internal directory to the order_gw service directory.

  1. Copy proto files.

Since the gRPC gateway service order_gw needs to know which API interfaces the order service provides, copy the order service's proto file. In the terminal, navigate to the order_gw directory and execute the command:

make copy-proto SERVER=../order
Enter fullscreen mode Exit fullscreen mode
  1. Open the configuration file configs/order_gw.yml and configure the order service address. 
grpcClient:
  - name: "order"
    host: "127.0.0.1"
    port: 8282
    registryDiscoveryType: ""
    enableLoadBalance: false
Enter fullscreen mode Exit fullscreen mode
  1. Generate and automatically merge API-related code. 
make proto
Enter fullscreen mode Exit fullscreen mode
  1. Fill in the business logic code, which involves converting HTTP requests into gRPC requests. Here, you can directly use the generated template code example. Click to view the code internal/service/order_gw.go.

After configuring and filling in the business logic code, compile and start the gRPC gateway service order_gw:

make run
Enter fullscreen mode Exit fullscreen mode

Testing Distributed Transactions

Open the swagger interface http://localhost:8080/apis/swagger/index.html in the browser to test the order submission API interface.

On the DTM management interface http://localhost:36789, you can view the status and details of distributed transactions,

Log messages can be viewed at the endpoints of each service to understand the status of the api interfaces that dtm orchestrates calls to.

Test Successful Order Submission Scenario

On the swagger interface, fill in the request parameters.

Image description

Click the Execute button to test. The order is successfully submitted, which can be seen from the DTM management interface and the logs of various services.

Test Failed Order Submission Scenario

  1. The order fails due to an invalid coupon.

With the request parameters unchanged,

{
  "userId": 1,
  "productId": 1,
  "amount": 1100,
  "productCount": 1,
  "couponId": 1
}
Enter fullscreen mode Exit fullscreen mode

Click the Execute button directly for testing. Although an order ID is returned (this does not mean the order is successful, you actually need to get the successful order status before performing subsequent operations), you can see from the DTM management interface and the coupon service log that the order status is failed because the coupon has been used. It returned an Aborted error. After DTM receives the Aborted error message, it will compensate for the order creation and coupon deduction branch transactions to ensure data consistency.

  1. The order fails due to insufficient inventory.

Fill in the request parameters, the value of the productCount field is 1000, which is definitely greater than the inventory quantity, and set the couponId parameter to 0 to indicate that no coupon is used.

{
  "userId": 1,
  "productId": 1,
  "amount": 1100000,
  "productCount": 1000,
  "couponId": 0
}
Enter fullscreen mode Exit fullscreen mode

Click the Execute button to test. Although an order ID is returned (this does not mean the order is successful), you can see from the DTM management interface and the inventory service log that the order status is failed due to insufficient inventory. It returned an Aborted error. After DTM receives the Aborted error message, it will compensate for the order creation branch transaction to ensure data consistency.

Test Simulated Process Crash, Successful Order Submission After Recovery Scenario

Stop the inventory service stock, then fill in the request parameters on the swagger interface:

{
  "userId": 1,
  "productId": 1,
  "amount": 1100,
  "productCount": 1,
  "couponId": 0
}
Enter fullscreen mode Exit fullscreen mode

Click the Execute button to test. Although an order ID is returned (this does not mean the order is successful), you can see from the DTM management interface that the order status is submitted. DTM will keep trying to connect to the inventory service stock. The retry is by default an exponential backoff algorithm. At this time, start the inventory service. After DTM connects to the inventory service, it completes the subsequent branch transactions and finally successfully completes the order, ensuring data consistency. Of course, you can also actively force the cancellation of the order on the business side when it times out. After DTM receives the forced cancellation of the order, it will compensate for the order creation branch transaction to ensure data consistency.

Summary

This article introduces the quick construction of a simple order system from scratch. Using Sponge makes it easy to build microservices, and DTM elegantly solves distributed transaction issues related to orders. Developing an order system becomes straightforward, allowing developers to spend their efforts on business development.

Each service comes with commonly used service governance features, such as service registration and discovery, rate limiting, circuit breaking, distributed tracing, monitoring, performance analysis, resource statistics, CICD, etc. These features can be uniformly enabled or disabled in the YAML configuration file.

Of course, this is not a complete order system; it only covers the simple business logic of submitting orders. If you need to build your own order system, this can serve as a reference. Following the above steps, it's easy to add modules like product service, logistics service, user service, etc., to create an e-commerce platform.

Top comments (0)

Read next

johnscode profile image

Sending IoT Device Data via MQTT broker.

J Fowler -

pradumnasaraf profile image

How to Publish a Golang Package

Pradumna Saraf -

ngocphuongnb profile image

Introducing FastSchema: A Powerful, Go-Based Web Framework & Headless CMS

Phuong Nguyen -

ortizdavid profile image

Como executar várias operações de forma transacional em Golang?

Ortiz de Arcanjo António David -