Truong Phung

Posted on Oct 25 • Edited on Dec 19

🧪 Golang Integration Test With Gin, Gorm, Testify, PostgreSQL

#webdev #go #postgres #testing

Creating a comprehensive integration test setup in Golang with Gin, GORM, Testify, and PostgreSQL involves setting up a test database, writing tests for CRUD operations, and using Testify for assertions. Here’s a step-by-step guide to get you started:

Prerequisites

Go installed
Docker installed
Libraries: gin-gonic/gin, gorm.io/gorm, gorm.io/driver/postgres, testify, testcontainers-go

Project Structure

myapp/
|-- main.go
|-- models/
|   |-- models.go
|-- handlers/
|   |-- handlers.go
|-- tests/
|   |-- integration_test.go
|-- go.mod
|-- go.sum

1. Setup the Models (`models/models.go`)

Define the models with GORM tags for database mapping.

package models

import (
    "time"
    "gorm.io/gorm"
)

type User struct {
    ID        uint           `gorm:"primaryKey"`
    Name      string         `gorm:"not null"`
    Email     string         `gorm:"unique;not null"`
    CreatedAt time.Time
}

type Book struct {
    ID            uint           `gorm:"primaryKey"`
    Title         string         `gorm:"not null"`
    Author        string         `gorm:"not null"`
    PublishedDate time.Time      `gorm:"not null"`
}

type BorrowLog struct {
    ID         uint           `gorm:"primaryKey"`
    UserID     uint           `gorm:"not null"`
    BookID     uint           `gorm:"not null"`
    BorrowedAt time.Time      `gorm:"default:CURRENT_TIMESTAMP"`
    ReturnedAt *time.Time
}

2. Setup Handlers (`handlers/handlers.go`)

Define the routes and handlers for CRUD operations using Gin.

package handlers

import (
    "myapp/models"
    "net/http"

    "github.com/gin-gonic/gin"
    "gorm.io/gorm"
)

type Handler struct {
    DB *gorm.DB
}

func (h *Handler) CreateUser(c *gin.Context) {
    var user models.User
    if err := c.ShouldBindJSON(&user); err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
        return
    }

    if err := h.DB.Create(&user).Error; err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }

    c.JSON(http.StatusCreated, user)
}

func (h *Handler) GetUser(c *gin.Context) {
    var user models.User
    if err := h.DB.First(&user, c.Param("id")).Error; err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "User not found"})
        return
    }

    c.JSON(http.StatusOK, user)
}

func (h *Handler) UpdateUser(c *gin.Context) {
    var user models.User
    if err := h.DB.First(&user, c.Param("id")).Error; err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "User not found"})
        return
    }

    if err := c.ShouldBindJSON(&user); err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
        return
    }

    if err := h.DB.Save(&user).Error; err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }

    c.JSON(http.StatusOK, user)
}

func (h *Handler) DeleteUser(c *gin.Context) {
    if err := h.DB.Delete(&models.User{}, c.Param("id")).Error; err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
        return
    }

    c.JSON(http.StatusOK, gin.H{"message": "User deleted"})
}

3. Main Application (`main.go`)

Set up the database connection and routes.

package main

import (
    "myapp/handlers"
    "myapp/models"
    "github.com/gin-gonic/gin"
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
    "log"
    "os"
)

func main() {
    dsn := "host=localhost user=postgres password=yourpassword dbname=testdb port=5432 sslmode=disable"
    db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
    if err != nil {
        log.Fatalf("failed to connect to database: %v", err)
    }

    // Auto migrate the models
    db.AutoMigrate(&models.User{}, &models.Book{}, &models.BorrowLog{})

    h := handlers.Handler{DB: db}

    r := gin.Default()

    r.POST("/users", h.CreateUser)
    r.GET("/users/:id", h.GetUser)
    r.PUT("/users/:id", h.UpdateUser)
    r.DELETE("/users/:id", h.DeleteUser)

    r.Run(":8080")
}

4. Integration Test (`tests/integration_test.go`)

Use Testify for setting up and asserting test results.

For database we can use a Dockerized PostgreSQL instance for testing purposes, which is isolated and can be quickly torn down after tests. Here’s how to set it up in Golang using testcontainers-go:

Install testcontainers-go:

go get github.com/testcontainers/testcontainers-go

Following is the integration_test.go file that sets up a PostgreSQL container for testing:

package tests

import (
    "context"
    "fmt"
    "myapp/handlers"
    "myapp/models"
    "bytes"
    "encoding/json"
    "net/http"
    "net/http/httptest"
    "testing"
    "time"

    "github.com/gin-gonic/gin"
    "github.com/stretchr/testify/assert"
    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/wait"
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

var db *gorm.DB
var h *handlers.Handler

func setupTestDB() (func(), error){
    ctx := context.Background()

    // Create PostgreSQL container
    req := testcontainers.ContainerRequest{
        Image:        "postgres:latest",
        ExposedPorts: []string{"5432/tcp"},
        Env: map[string]string{
            "POSTGRES_PASSWORD": "password",
            "POSTGRES_DB":       "testdb",
        },
        WaitingFor: wait.ForListeningPort("5432/tcp").WithStartupTimeout(60 * time.Second),
    }

    postgresC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    })
    if err != nil {
        // panic(err)
        return nil, err
    }

    // Get the container's host and port
    host, _ := postgresC.Host(ctx)
    port, _ := postgresC.MappedPort(ctx, "5432")

    dsn := fmt.Sprintf("host=%s port=%s user=postgres password=password dbname=testdb sslmode=disable", host, port.Port())

    // Connect to the PostgreSQL database
    db, err = gorm.Open(postgres.Open(dsn), &gorm.Config{})
    if err != nil {
        // panic("failed to connect to database")
        return nil, err
    }

    // Migrate the schema
    db.AutoMigrate(&models.User{}, &models.Book{}, &models.BorrowLog{})

    // Initialize the handler
    h = &handlers.Handler{DB: db}

    // Clean up database before each test
    db.Exec("DELETE FROM users")

    // Tear down the container after tests
    // defer postgresC.Terminate(ctx)
    cleanup := func() {
        postgresC.Terminate(ctx)
    }

    return cleanup, nil
}

func TestCreateUser(t *testing.T) {

    // Set up test database
    cleanup, err := setupTestDB()
    if err != nil {
        t.Fatalf("failed to set up test DB: %v", err)
    }
    defer cleanup()

    r := gin.Default()
    r.POST("/users", h.CreateUser)

    user := models.User{
        Name:  "Test User",
        Email: "testuser@example.com",
    }

    jsonData, _ := json.Marshal(user)
    req, _ := http.NewRequest(http.MethodPost, "/users", bytes.NewBuffer(jsonData))
    req.Header.Set("Content-Type", "application/json")

    w := httptest.NewRecorder()
    r.ServeHTTP(w, req)

    assert.Equal(t, http.StatusCreated, w.Code)

    var createdUser models.User
    err := json.Unmarshal(w.Body.Bytes(), &createdUser)
    assert.Nil(t, err)
    assert.Equal(t, user.Name, createdUser.Name)
    assert.Equal(t, user.Email, createdUser.Email)
}

func TestGetUser(t *testing.T) {
    setupTestDB()

    // Create a user
    user := models.User{
        Name:  "Test User",
        Email: "testuser@example.com",
    }
    db.Create(&user)

    r := gin.Default()
    r.GET("/users/:id", h.GetUser)

    req, _ := http.NewRequest(http.MethodGet, fmt.Sprintf("/users/%d", user.ID), nil)
    w := httptest.NewRecorder()
    r.ServeHTTP(w, req)

    assert.Equal(t, http.StatusOK, w.Code)

    var fetchedUser models.User
    err := json.Unmarshal(w.Body.Bytes(), &fetchedUser)
    assert.Nil(t, err)
    assert.Equal(t, user.Name, fetchedUser.Name)
    assert.Equal(t, user.Email, fetchedUser.Email)
}

func TestUpdateUser(t *testing.T) {
    setupTestDB()

    // Create a user to be updated.
    user := models.User{
        Name:  "Original User",
        Email: "original@example.com",
    }
    db.Create(&user)

    r := gin.Default()
    r.PUT("/users/:id", h.UpdateUser)

    updatedData := models.User{
        Name:  "Updated User",
        Email: "updated@example.com",
    }

    jsonData, _ := json.Marshal(updatedData)
    req, _ := http.NewRequest(http.MethodPut, fmt.Sprintf("/users/%d", user.ID), bytes.NewBuffer(jsonData))
    req.Header.Set("Content-Type", "application/json")

    w := httptest.NewRecorder()
    r.ServeHTTP(w, req)

    assert.Equal(t, http.StatusOK, w.Code)

    var updatedUser models.User
    err := json.Unmarshal(w.Body.Bytes(), &updatedUser)
    assert.Nil(t, err)
    assert.Equal(t, updatedData.Name, updatedUser.Name)
    assert.Equal(t, updatedData.Email, updatedUser.Email)

    // Verify that the user is actually updated in the database.
    var userInDB models.User
    db.First(&userInDB, user.ID)
    assert.Equal(t, updatedData.Name, userInDB.Name)
    assert.Equal(t, updatedData.Email, userInDB.Email)
}

func TestDeleteUser(t *testing.T) {
    setupTestDB()

    // Create a user to be deleted.
    user := models.User{
        Name:  "Delete User",
        Email: "delete@example.com",
    }
    db.Create(&user)

    r := gin.Default()
    r.DELETE("/users/:id", h.DeleteUser)

    req, _ := http.NewRequest(http.MethodDelete, fmt.Sprintf("/users/%d", user.ID), nil)
    w := httptest.NewRecorder()
    r.ServeHTTP(w, req)

    assert.Equal(t, http.StatusOK, w.Code)

    // Verify the response message.
    var response map[string]string
    err := json.Unmarshal(w.Body.Bytes(), &response)
    assert.Nil(t, err)
    assert.Equal(t, "User deleted", response["message"])

    // Verify that the user is actually deleted from the database.
    var userInDB models.User
    result := db.First(&userInDB, user.ID)
    assert.Error(t, result.Error)
    assert.Equal(t, gorm.ErrRecordNotFound, result.Error)
}

Explanation

SetupTestDB: Sets up a PostgreSQL database connection using GORM for testing.
TestCreateUser: Sends a POST request to create a new user and asserts the response.
TestGetUser: Retrieves a user by ID and checks that the data matches what was inserted.
TestUpdateUser:
- Creates a user and updates it using the PUT /users/:id endpoint.
- Asserts that the response status is 200 OK.
- Verifies that the user's details are updated in the response.
- Fetches the user from the database and confirms that the changes are persisted.
TestDeleteUser:
- Creates a user and deletes it using the DELETE /users/:id endpoint.
- Asserts that the response status is 200 OK and checks for a success message.
- Attempts to fetch the deleted user from the database to ensure the user no longer exists, asserting an error of gorm.ErrRecordNotFound.
testcontainers-go: This library allows you to spin up Docker containers directly from your Go code. It's ideal for creating a temporary PostgreSQL instance for integration tests.
setupTestDB: This function starts a PostgreSQL Docker container, connects to it using gorm, and sets up the database schema. It also ensures that the container is cleaned up after the tests are finished.
defer postgresC.Terminate(ctx): Ensures that the PostgreSQL container is terminated after tests are done, simulating an in-memory approach.
Dynamic Host and Port: Uses the container's dynamically allocated host and port for connecting to the database.

Running the Tests

Run the tests using:

go test ./tests -v

Benefits of Using `testcontainers-go`:

Isolation: Each test run gets a fresh PostgreSQL instance, ensuring no data leakage between tests.
Replicates Production Environment: Testing against a real PostgreSQL instance provides more reliable results than using an in-memory database.
Automation: Automatically starts and stops the PostgreSQL container, making it easy to use in CI/CD pipelines.

Key Points

Using a Test Database: It's a good practice to use a separate PostgreSQL database (ex: containerized ones) for testing to avoid affecting production data.
Setup and Cleanup: Ensure to clean up the database between tests to maintain consistency.
Testify: Provides powerful assertion methods for validating the results.
Gin's Test Server: Uses httptest for simulating HTTP requests against the Gin server.

With this setup, you can test CRUD operations for a User model, ensuring the API works as expected with PostgreSQL. You can expand the tests similarly for Book and BorrowLog models.

5. Integration Tests With Migrations. (extension part)

Using golang-migrate with Testcontainers and httptest in integration tests is a practical approach to manage your test database schema. Here's how you can set it up step by step:

1. Start a Test Database with Testcontainers

Use the testcontainers-go library to spin up a containerized database (e.g., PostgreSQL or MySQL) during your integration tests.

Example (for PostgreSQL):

package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"
    "testing"
    "time"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/wait"
    _ "github.com/lib/pq" // PostgreSQL driver
)

func setupTestDB() (*sql.DB, string, func(), error) {
    ctx := context.Background()

    // Create a PostgreSQL container
    req := testcontainers.ContainerRequest{
        Image:        "postgres:15-alpine",
        ExposedPorts: []string{"5432/tcp"},
        Env: map[string]string{
            "POSTGRES_USER":     "test",
            "POSTGRES_PASSWORD": "test",
            "POSTGRES_DB":       "testdb",
        },
        WaitingFor: wait.ForSQL("5432/tcp", "postgres", func(port nat.Port) string {
            return fmt.Sprintf("host=localhost port=%s user=test password=test dbname=testdb sslmode=disable", port.Port())
        }),
    }

    container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    })
    if err != nil {
        return nil, "", nil, err
    }

    // Get the container's host and port
    host, err := container.Host(ctx)
    if err != nil {
        return nil, "", nil, err
    }

    port, err := container.MappedPort(ctx, "5432/tcp")
    if err != nil {
        return nil, "", nil, err
    }

    // Create a database connection string
    dsn := fmt.Sprintf("host=%s port=%s user=test password=test dbname=testdb sslmode=disable", host, port.Port())

    // Open the database connection
    db, err := sql.Open("postgres", dsn)
    if err != nil {
        return nil, "", nil, err
    }

    // Return a cleanup function to stop the container
    cleanup := func() {
        container.Terminate(ctx)
    }

    return db, dsn, cleanup, nil
}

2. Apply Migrations with golang-migrate

Once the test database is up, use golang-migrate to apply your migrations.

Install golang-migrate library:

go get -u github.com/golang-migrate/migrate/v4

Example (applying migrations):

import (
    "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database/postgres"
    "github.com/golang-migrate/migrate/v4/source/file"
)

func applyMigrations(dsn string) error {
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    if err != nil {
        return err
    }

    m, err := migrate.NewWithDatabaseInstance(
        "file://path/to/migrations", // Migration files location
        "postgres",                 // Database name
        driver,
    )
    if err != nil {
        return err
    }

    err = m.Up() // Apply all migrations
    if err != nil && err != migrate.ErrNoChange {
        return err
    }

    return nil
}

3. Use `httptest` for Integration Tests

Use httptest to create an HTTP server for your integration tests.

Example:

import (
    "net/http"
    "net/http/httptest"
)

func TestIntegration(t *testing.T) {
    // Set up test database
    db, dsn, cleanup, err := setupTestDB()
    if err != nil {
        t.Fatalf("failed to set up test DB: %v", err)
    }
    defer cleanup()

    // Apply migrations
    if err := applyMigrations(dsn); err != nil {
        t.Fatalf("failed to apply migrations: %v", err)
    }

    // Initialize your application with the test database
    app := NewApp(db) // Assume NewApp initializes your app

    // Create an HTTP test server
    server := httptest.NewServer(app.Router) // Assume Router is your HTTP router
    defer server.Close()

    // Perform test requests
    resp, err := http.Get(server.URL + "/some-endpoint")
    if err != nil {
        t.Fatalf("failed to make GET request: %v", err)
    }
    defer resp.Body.Close()

    // Assert response
    if resp.StatusCode != http.StatusOK {
        t.Fatalf("expected status OK, got %v", resp.StatusCode)
    }
}

4. Best Practices

Use Isolated Test Containers: Each test should spin up its own database container to avoid interference.
Clean Up After Tests: Ensure containers and test resources are properly terminated after tests.
Seed Test Data: Use migrations or direct SQL inserts to seed data needed for the tests.
Parallel Tests: If running tests in parallel, ensure separate containers or databases are used for isolation.

This approach provides a clean, reliable way to test your application with real database interactions and migrations while leveraging Testcontainers and httptest.

If you found this helpful, let me know by leaving a 👍 or a comment!, or if you think this post could help someone, feel free to share it! Thank you very much! 😃

DEV Community

🧪 Golang Integration Test With Gin, Gorm, Testify, PostgreSQL

Prerequisites

Project Structure

1. Setup the Models (`models/models.go`)

2. Setup Handlers (`handlers/handlers.go`)

3. Main Application (`main.go`)

4. Integration Test (`tests/integration_test.go`)

Explanation

Running the Tests

Benefits of Using `testcontainers-go`:

Key Points

5. Integration Tests With Migrations. (extension part)

1. Start a Test Database with Testcontainers

2. Apply Migrations with golang-migrate

3. Use `httptest` for Integration Tests

4. Best Practices

Top comments (0)

Read next

Why Offshore Laravel Development in Vietnam is a Smart Choice for Your Business

HTTPS for Django Development Environment

iter.json: A Powerful and Efficient Way to Iterate and Manipulate JSON in Go

Override Go app configuration with Environment variable

Prerequisites

Project Structure

1. Setup the Models (models/models.go)

2. Setup Handlers (handlers/handlers.go)

3. Main Application (main.go)

4. Integration Test (tests/integration_test.go)

Explanation

Running the Tests

Benefits of Using testcontainers-go:

Key Points

5. Integration Tests With Migrations. (extension part)

1. Start a Test Database with Testcontainers

2. Apply Migrations with golang-migrate

3. Use httptest for Integration Tests

4. Best Practices

Read next

Why Offshore Laravel Development in Vietnam is a Smart Choice for Your Business

HTTPS for Django Development Environment

iter.json: A Powerful and Efficient Way to Iterate and Manipulate JSON in Go

Override Go app configuration with Environment variable

1. Setup the Models (`models/models.go`)

2. Setup Handlers (`handlers/handlers.go`)

3. Main Application (`main.go`)

4. Integration Test (`tests/integration_test.go`)

Benefits of Using `testcontainers-go`:

3. Use `httptest` for Integration Tests