Gorm: Sneak Peek of Custom Data Types

Welcome back, folks 👀! Today, we discuss a specific use case we might face when moving data back and forth from/to the database. First, let me set the boundaries for today's challenge. To stick to a real-life example, let's borrow some concepts from the U.S. Army 🪖. Our deal is to write a small software to save and read the officers with the grades they have achieved in their careers.

Gorm's Custom Data Types

Our software needs to handle the army officers with their respective grades. At first look, it might seem easy, and we probably don't need any Custom Data Type here. However, to show off this feature, let's use a non-conventional way to represent the data. Thanks to this, we're asked to define a custom mapping between Go structs and DB relations. Furthermore, we must define a specific logic to parse the data. Let's expand on this by looking at the program's targets 🌏.

Use Case to Handle

To ease things down, let's use a drawing to depict the relationships between the code and the SQL objects:

Let's focus on each container one by one.

The Go Structs 🧱

Here, we defined two structs. The Grade struct holds a non-exhaustive list of military grades 🎖️. This struct won't be a table in the database. Conversely, the Officer struct contains the ID, the name, and a pointer to the Grade struct, indicating which grades have been achieved by the officer so far.

Whenever we write an officer to the DB, the column grades_achieved must contain an array of strings filled in with the grades achieved (the ones with true in the Grade struct).

The DB Relations 🙌

Regarding the SQL objects, we have only the officers table. The id and name columns are self-explanatory. Then, we have the grades_achieved column that holds the officer's grades in a collection of strings.

Whenever we decode an officer from the database, we parse the grades_achieved column and create a matching "instance" of the Grade struct.

You might have noticed that the behavior is not the standard one. We must make some arrangements to fulfill it in the desired way.

Here, the models' layout is overcomplicated on purpose. Please stick to more straightforward solutions whenever possible.

Custom Data Types

Gorm provides us with Custom Data Types. They give us great flexibility in defining the retrieval and saving to/from the database. We must implement two interfaces: Scanner and Valuer 🤝. The former specifies a custom behavior to apply when fetching data from the DB. The latter indicates how to write values in the database. Both help us in achieving the non-conventional mapping logic we need.

The functions' signatures we must implement are Scan(value interface{}) error and Value() (driver.Value, error). Now, let's look at the code.

The Code

The code for this example lives in two files: the domain/models.go and the main.go. Let's start with the first one, dealing with the models (translated as structs in Go).

The domain/models.go file

First, let me present the code for this file:

package models

import (
 "database/sql/driver"
 "slices"
 "strings"
)

type Grade struct {
 Lieutenant bool
 Captain    bool
 Colonel    bool
 General    bool
}

type Officer struct {
 ID             uint64 `gorm:"primaryKey"`
 Name           string
 GradesAchieved *Grade `gorm:"type:varchar[]"`
}

func (g *Grade) Scan(value interface{}) error {
 // we should have utilized the "comma, ok" idiom
 valueRaw := value.(string)
 valueRaw = strings.Replace(strings.Replace(valueRaw, "{", "", -1), "}", "", -1)
 grades := strings.Split(valueRaw, ",")
 if slices.Contains(grades, "lieutenant") {
 g.Lieutenant = true
 }
 if slices.Contains(grades, "captain") {
 g.Captain = true
 }
 if slices.Contains(grades, "colonel") {
 g.Colonel = true
 }
 if slices.Contains(grades, "general") {
 g.General = true
 }
 return nil
}

func (g Grade) Value() (driver.Value, error) {
 grades := make([]string, 0, 4)
 if g.Lieutenant {
 grades = append(grades, "lieutenant")
 }
 if g.Captain {
 grades = append(grades, "captain")
 }
 if g.Colonel {
 grades = append(grades, "colonel")
 }
 if g.General {
 grades = append(grades, "general")
 }
 return grades, nil
}

Now, let's highlight the relevant parts of it 🔎:

1. The Grade struct only lists the grades we forecasted in our software
2. The Officer struct defines the characteristics of the entity. This entity is a relation in the DB. We applied two Gorm notations:
   1. gorm:"primaryKey" on the ID field to define it as the primary key of our relation
   2. gorm:"type:varchar[]" to map the field GradesAchieved as an array of varchar in the DB. Otherwise, it translates as a separate DB table or additional columns in the officers table
3. The Grade struct implements the Scan function. Here, we get the raw value, we adjust it, we set some fields on the g variable, and we return
4. The Grade struct also implements the Value function as a value receiver type (we don't need to change the receiver this time, we don't use the * reference). We return the value to write in the column grades_achieved of the officers table

Thanks to these two methods, we can control how to send and retrieve the type Grade during DB interactions. Now, let's look at the main.go file.

The main.go file 🚦

Here, we prepare the DB connection, migrate the objects to relations (ORM stands for Object Relation Mapping), and insert and fetch records to test the logic. Below is the code:

package main

import (
 "encoding/json"
 "fmt"
 "os"

 "gormcustomdatatype/models"

 "gorm.io/driver/postgres"
 "gorm.io/gorm"
)

func seedDB(db *gorm.DB, file string) error {
 data, err := os.ReadFile(file)
 if err != nil {
  return err
 }
 if err := db.Exec(string(data)).Error; err != nil {
  return err
 }
 return nil
}

// docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
func main() {
 dsn := "host=localhost port=54322 user=postgres password=postgres dbname=postgres sslmode=disable"
 db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
 if err != nil {
 fmt.Fprintf(os.Stderr, "could not connect to DB: %v", err)
  return
 }
 db.AutoMigrate(&models.Officer{})
 defer func() {
 db.Migrator().DropTable(&models.Officer{})
 }()
 if err := seedDB(db, "data.sql"); err != nil {
 fmt.Fprintf(os.Stderr, "failed to seed DB: %v", err)
  return
 }
 // print all the officers
 var officers []models.Officer
 if err := db.Find(&officers).Error; err != nil {
 fmt.Fprintf(os.Stderr, "could not get the officers from the DB: %v", err)
  return
 }
 data, _ := json.MarshalIndent(officers, "", "\t")
 fmt.Fprintln(os.Stdout, string(data))

 // add a new officer
 db.Create(&models.Officer{
 Name: "Monkey D. Garp",
 GradesAchieved: &models.Grade{
 Lieutenant: true,
 Captain:    true,
 Colonel:    true,
 General:    true,
  },
 })
 var garpTheHero models.Officer
 if err := db.First(&garpTheHero, 4).Error; err != nil {
 fmt.Fprintf(os.Stderr, "failed to get officer from the DB: %v", err)
  return
 }
 data, _ = json.MarshalIndent(&garpTheHero, "", "\t")
 fmt.Fprintln(os.Stdout, string(data))
}

Now, let's see the relevant sections of this file. First, we define the seedDB function to add dummy data in the DB. The data lives in the data.sql file with the following content:

INSERT INTO public.officers
(id, "name", grades_achieved)
VALUES(nextval('officers_id_seq'::regclass), 'john doe', '{captain,lieutenant}'),
(nextval('officers_id_seq'::regclass), 'gerard butler', '{general}'),
(nextval('officers_id_seq'::regclass), 'chuck norris', '{lieutenant,captain,colonel}');

The main() function starts by setting up a DB connection. For this demo, we used PostgreSQL. Then, we ensure the officers table exists in the database and is up-to-date with the newest version of the models.Officer struct. Since this program is a sample, we did two additional things:

• Removal of the table at the end of the main() function (when the program terminates, we would like to remove the table as well)
• Seeding of some dummy data

Lastly, to ensure that everything works as expected, we do a couple of things:

1. Fetching all the records in the DB
2. Adding (and fetching back) a new officer

That's it for this file. Now, let's test our work 📸.

The Truth Moment

Before running the code, please ensure that a PostgreSQL instance is running on your machine. With Docker 🐋, you can run this command:

docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres

Now, we can safely run our application by issuing the command: go run . 🚀

The output is:

[
        {
                "ID": 1,
                "Name": "john doe",
                "GradesAchieved": {
                        "Lieutenant": true,
                        "Captain": true,
                        "Colonel": false,
                        "General": false
                }
        },
        {
                "ID": 2,
                "Name": "gerard butler",
                "GradesAchieved": {
                        "Lieutenant": false,
                        "Captain": false,
                        "Colonel": false,
                        "General": true
                }
        },
        {
                "ID": 3,
                "Name": "chuck norris",
                "GradesAchieved": {
                        "Lieutenant": true,
                        "Captain": true,
                        "Colonel": true,
                        "General": false
                }
        }
]
{
        "ID": 4,
        "Name": "Monkey D. Garp",
        "GradesAchieved": {
                "Lieutenant": true,
                "Captain": true,
                "Colonel": true,
                "General": true
        }
}

Voilà! Everything works as expected. We can re-run the code several times and always have the same output.

That's a Wrap

I hope you enjoyed this blog post regarding Gorm and the Custom Data Types. I always recommend you stick to the most straightforward approach. Opt for this only if you eventually need it. This approach adds flexibility in exchange for making the code more complex and less robust (a tiny change in the structs' definitions might lead to errors and extra work needed).

Keep this in mind. If you stick to conventions, you can be less verbose throughout your codebase.

That's a great quote to end this blog post.
If you realize that Custom Data Types are needed, this blog post should be a good starting point to present you with a working solution.

Please let me know your feelings and thoughts. Any feedback is always appreciated! If you're interested in a specific topic, reach out, and I'll shortlist it. Until next time, stay safe, and see you soon!