Lorain

Posted on Nov 13, 2022

High Performance Web Framework Tasting-Database Operations

#webdev #beginners #tutorial #go

Introduction

In the previous post, we gave a brief introduction to the high-performance Go HTTP framework Hertz and completed a simple demo using Hertz to get you started.

In this post, you'll learn more about using the Hertz framework with an official demo.

And we'll highlight the following features:

Use thrift IDL to define HTTP interface
Use hz to generate code
Use Hertz binding and validate
Use GORM and MySQL

Installation

Run the following command to get the official demo:

git clone https://github.com/cloudwego/hertz-examples.git
cd bizdemo/hertz_gorm

Project Structure

hertz_gorm
├── biz
|   ├── dal             // Logic code that interacts with the database
│   ├── handler         // Main logical code that handles HTTP requests                  
│   ├── hertz_gen       // Scaffolding generated by hertz from idl files
|   ├── model           // Go struct corresponding to the database table
|   ├── pack            // Transformation between database model and response model
|   ├── router          // Middleware and mapping of routes to handlers
├── go.mod              // go.mod                 
├── idl                 // thift idl                  
├── main.go             // Initialize and start the server              
├── router.go           // Sample route registration
├── router_gen.go       // Route registration    
├── docker-compose.yml  // docker-compose.yml
├── Makefile            // Makefile

This is the basic architecture for the project. It's pretty clean and simple, and hz generated a lot of scaffolding code for us as well.

Define IDL

hz is a tool provided by the Hertz framework for generating code. Currently, hz can generate scaffolding for Hertz projects based on thrift and protobuf IDL.

The definition of an excellent IDL file plays an important role in developing with Hertz. We will use the thrift IDL for this project as an example.

We can use api annotations to let hz help us with parameter binding and validation, route registration code generation, etc.

hz will generate the go tag based on the following api annotations so that Hertz can retrieve these values using reflection and parse them.

Field Annotation

The go-tagexpr open source library is used for parameter binding and validation of the Field annotation, as shown in the following example for CreateUserRequest:

// api.thrift
struct CreateUserRequest{
    1: string name      (api.body="name", api.form="name",api.vd="(len($) > 0 && len($) < 100)")
    2: Gender gender    (api.body="gender", api.form="gender",api.vd="($ == 1||$ == 2)")
    3: i64    age       (api.body="age", api.form="age",api.vd="$>0")
    4: string introduce (api.body="introduce", api.form="introduce",api.vd="(len($) > 0 && len($) < 1000)")
}

The form annotation allows hz to automatically bind the parameters in the form of an HTTP request body for us, saving us the trouble of manually binding them using methods such as PostForm.

The vd annotation allows for parameter validation. For example, CreateUserRequest uses the vd annotation to ensure that the gender field is only 1 or 2.

You may refer to here for more information about parameter validation syntax.

Method Annotation

The Method annotation can be used to generate route registration code.

Consider the following UserService:

// api.thrift
service UserService {
   UpdateUserResponse UpdateUser(1:UpdateUserRequest req)(api.post="/v1/user/update/:user_id")
   DeleteUserResponse DeleteUser(1:DeleteUserRequest req)(api.post="/v1/user/delete/:user_id")
   QueryUserResponse  QueryUser(1: QueryUserRequest req)(api.post="/v1/user/query/")
   CreateUserResponse CreateUser(1:CreateUserRequest req)(api.post="/v1/user/create/")
}

We defined POST methods and routes using post annotations, and hz will generate handler methods for each route as well as route grouping, middleware embedding scaffolding, etc. As shown in biz/router/user_gorm/api.go and biz/handler/user_gorm/user_service.go.

And we can also define the business error code in the idl file:

// api.thrift
enum Code {
     Success         = 1
     ParamInvalid    = 2
     DBErr           = 3
}

hz will generate constants and related methods for us based on these.

// biz/hertz_gen/user_gorm/api.go
type Code int64

const (
    Code_Success      Code = 1
    Code_ParamInvalid Code = 2
    Code_DBErr        Code = 3
)

Generate Code with hz

After we finish writing IDL, we can generate the scaffolding code for us by using hz.

Execute the following command to generate code:

hz new --model_dir biz/hertz_gen -mod github.com/cloudwego/hertz-examples/bizdemo/hertz_gorm -idl idl/api.thrift

Execute the following command to update the code if you edit the IDL after the first generated:

hz update --model_dir biz/hertz_gen -idl idl/api.thrift

Of course, the project has already generated the code for you, so you don't need to execute it. When you actually use Hertz for web development yourself, I'm sure you'll find it a very efficient and fun tool.

Use Middleware

In this project, we configured the root route group to use the gzip middleware for all routes to improve performance.

// biz/router/user_gorm/middleware.go
func rootMw() []app.HandlerFunc {
    // your code...
    // use gzip middleware
    return []app.HandlerFunc{gzip.Gzip(gzip.DefaultCompression)}
}

Just add one line of code to the generated scaffolding code, very easy. You can also refer to the hertz-contrib/gzip for more custom configuration.

Manipulating database with GORM

Configure GORM

To use GORM with a database, you first need to connect to the database using a driver and configure GORM, as shown in biz/dal/mysql/init.go.

// biz/dal/mysql/user.go
package mysql

import (
   "gorm.io/driver/mysql"
   "gorm.io/gorm"
   "gorm.io/gorm/logger"
)

var dsn = "gorm:gorm@tcp(localhost:9910)/gorm?charset=utf8&parseTime=True&loc=Local"

var DB *gorm.DB

func Init() {
   var err error
   DB, err = gorm.Open(mysql.Open(dsn), &gorm.Config{
      SkipDefaultTransaction: true,
      PrepareStmt:            true,
      Logger:                 logger.Default.LogMode(logger.Info),
   })
   if err != nil {
      panic(err)
   }
}

Here we connect with MySQL database by means of DSN and maintain a global database operation object DB.

In terms of GORM configuration, since this project does not involve the operation of multiple tables at the same time, we can configure SkipDefaultTransaction to true to skip the default transaction, and enable caching through PrepareStmt to improve efficiency.

We also use the default logger so that we can clearly see the SQL generated for us by GORM.

Manipulating MySQL

GORM concatenates SQL statements to perform CRUD, so the code is very concise and easy to read, where all the database operations are in biz/dal/mysql/user.go.

We also declare a model corresponding to the database table, the gorm.Model contains some common fields, which GORM can automatically fill in for us, and support operations such as soft deletion.

// biz/model/user.go
type User struct {
   gorm.Model
   Name      string `json:"name" column:"name"`
   Gender    int64  `json:"gender" column:"gender"`
   Age       int64  `json:"age" column:"age"`
   Introduce string `json:"introduce" column:"introduce"`
}

Handle HTTP Request

In this section, we'll explore the handler (biz/handler/user_gorm/user_service.go), which is the main business logic code.

CreateUser & DeleteUser & UpdateUser

CreateUser

Since we are using api annotations in the thift IDL, BindAndValidate will do the parameter binding and validation for us . Very conveniently, all valid parameters will be injected into CreateUserRequest.

If there is an error, we can use the JSON method to return the data in JSON format . Whether it is CreateUserResponse or the business code, we can directly use the code generated by hz.

After that, we can insert a new user into MySQL by calling the CreateUser in the dal layer, passing in the encapsulated arguments.

If there is an error, we return JSON with the error code and information, just like we did in the beginning. Otherwise, the correct service code is returned to represent the successful creation of the user.

// biz/handler/user_gorm/user_service.go
// CreateUser .
// @router /v1/user/create/ [POST]
func CreateUser(ctx context.Context, c *app.RequestContext) {
   var err error
   var req user_gorm.CreateUserRequest
   err = c.BindAndValidate(&req)
   if err != nil {
      c.JSON(200, &user_gorm.CreateUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
      return
   }
   if err = mysql.CreateUser([]*model.User{
      {
         Name:      req.Name,
         Gender:    int64(req.Gender),
         Age:       req.Age,
         Introduce: req.Introduce,
      },
   }); err != nil {
      c.JSON(200, &user_gorm.CreateUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
      return
   }

   resp := new(user_gorm.CreateUserResponse)
   resp.Code = user_gorm.Code_Success
   c.JSON(200, resp)
}

DeleteUser

The logic for DeleteUser and CreateUser is almost identical: Bind and validate the arguments, use mysql.DeleteUser to delete the user, and return if there is an error, otherwise, return success.

// biz/handler/user_gorm/user_service.go
// DeleteUser .
// @router /v1/user/delete/:user_id [POST]
func DeleteUser(ctx context.Context, c *app.RequestContext) {
   var err error
   var req user_gorm.DeleteUserRequest
   err = c.BindAndValidate(&req)
   if err != nil {
      c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
      return
   }
   if err = mysql.DeleteUser(req.UserID); err != nil {
      c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
      return
   }

   c.JSON(200, &user_gorm.DeleteUserResponse{Code: user_gorm.Code_Success})
}

UpdateUser

UpdateUser is much the same, with the notable model transformation from an object that receives HTTP request parameters to a data access object that corresponds to a database table.

// biz/handler/user_gorm/user_service.go
// UpdateUser .
// @router /v1/user/update/:user_id [POST]
func UpdateUser(ctx context.Context, c *app.RequestContext) {
    var err error
    var req user_gorm.UpdateUserRequest
    err = c.BindAndValidate(&req)
    if err != nil {
            c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
            return
    }

    u := &model.User{}
    u.ID = uint(req.UserID)
    u.Name = req.Name
    u.Gender = int64(req.Gender)
    u.Age = req.Age
    u.Introduce = req.Introduce

    if err = mysql.UpdateUser(u); err != nil {
            c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
            return
    }

    c.JSON(200, &user_gorm.UpdateUserResponse{Code: user_gorm.Code_Success})
}

QueryUser

What's worth noting in QueryUser is that we're doing paging and a transformation from model.User to user_gorm.User, which is the reverse of the operation we just mentioned in UpdateUser.

With a simple paging formula startIndex = (currentPage - 1) * pageSize, we're paging the data as we're querying it.

And this time we've wrapped our transformation model in biz/pack/user.go.

// biz/pack/user.go
// Users Convert model.User list to user_gorm.User list
func Users(models []*model.User) []*user_gorm.User {
    users := make([]*user_gorm.User, 0, len(models))
    for _, m := range models {
            if u := User(m); u != nil {
                    users = append(users, u)
            }
    }
    return users
}

// User Convert model.User to user_gorm.User
func User(model *model.User) *user_gorm.User {
    if model == nil {
            return nil
    }
    return &user_gorm.User{
            UserID:    int64(model.ID),
            Name:      model.Name,
            Gender:    user_gorm.Gender(model.Gender),
            Age:       model.Age,
            Introduce: model.Introduce,
    }
}
// biz/handler/user_gorm/user_service.go
// QueryUser .
// @router /v1/user/query/ [POST]
func QueryUser(ctx context.Context, c *app.RequestContext) {
    var err error
    var req user_gorm.QueryUserRequest
    err = c.BindAndValidate(&req)
    if err != nil {
            c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_ParamInvalid, Msg: err.Error()})
            return
    }

    users, total, err := mysql.QueryUser(req.Keyword, req.Page, req.PageSize)
    if err != nil {
            c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_DBErr, Msg: err.Error()})
            return
    }
    c.JSON(200, &user_gorm.QueryUserResponse{Code: user_gorm.Code_Success, Users: pack.Users(users), Totoal: total})
}

The rest of the business logic is the same as before, and we're done with all the handler functions.

Run Demo

Run mysql docker

cd bizdemo/hertz_gorm && docker-compose up

Generate MySQL table

Connect MySQL and execute user.sql

Run demo

cd bizdemo/hertz_gorm
go build -o hertz_gorm && ./hertz_gorm

Summary

That's it for this post. Hopefully it will give you a quick overview of how to develop with Hertz and GORM. Both of them are well documented . Feel free to check out the official documentation for more information.

DEV Community