Hello everyone! Let’s start the hands-on section of the gRPC course. The target of the whole section is to build a "pc book" web service that will allow us to manage and search for laptop configurations.

Here's the link to the full gRPC course playlist on Youtube
Github repository: pcbook-go and pcbook-java
Gitlab repository: pcbook-go and pcbook-java

Protocol buffer basics

In this lecture, we will learn how to write a simple protocol-buffer message with some basic data types, install Visual Studio Code plugins to work with protobuf, and finally we will install protocol-buffer compiler and write a Makefile to run code generation for Go.

But before start, make sure that you already have Go and Visual Studio Code up and running properly on your computer. If not, you can watch my tutorial video on how to install Go and setup Visual Studio code:

The tutorial will guide you, step by step, to install Go, add the bin folder to your PATH, install Visual Studio Code, customise its theme and setup Go extensions to work with it.

Once everything is ready, you can come back here and continue this lecture.

Install vscode plugins

Alright, let's start by creating a new project. First, I will create a simple hello-world program in main.go file and run it, just to make sure that Go is working properly.

package main

import "fmt"

func main() {
    fmt.Println("Hello world")
}

Then create a new folder named protoc, and add a processor_message.proto file under it.

pcbook
├── proto
│   └── processor_message.proto
└── main.go

Vscode will ask us to install the extensions for the proto file. So let's go to the marketplace and search for ext:proto

There are 2 extensions shown at the top that we should install: clang-format and vscode-proto3. Let’s click install for both of them.

How to define a protobuf message

Now come back to our proto file. This file will contain the message definition of the CPU of a laptop.

We start with syntax = "proto3".

At the moment, there are 2 versions of protocol buffer on Google's official documentation: proto2 and proto3. For simplicity, we will only use proto3 (the newer version) in this course.

The syntax is pretty simple, just use the message keyword followed by the name of the message. Then inside the message block, we define all of its fields as shown in this picture:

Note that the name of the message should be UpperCamelCase, and the name of the field should be lower_snake_case.

There are many built-in scalar-value data types that we can use, for instance: string, bool, byte, float, double, and many other integer types. We can also use our own data types, such as enums or other messages.

Each message field should be assigned a unique tag. And the tag is more important than the field name because protobuf will use it to serialise the message.

A tag is simply an arbitrary integer with the smallest value of 1, and the biggest value of 2²⁹ - 1, except for numbers from 19000 to 19999, as they're reserved for internal protocol buffers implementation.

Note that tags from 1 to 15 take only 1 byte to encode, while those from 16 to 2047 take 2 bytes. So you should use them wisely, like: saving tags from 1 to 15 for very frequently occurring fields.

And remember that the tags don't need to be in-order (or sequential), but they must be unique for the same-level fields of the message.

Define the CPU message

Now let's get back to our proto file and define the CPU message.

syntax = "proto3";

message CPU {
  string brand = 1;
  string name = 2;
  uint32 number_cores = 3;
  uint32 number_threads = 4;
  double min_ghz = 5;
  double max_ghz = 6;
}

The CPU will have a brand of type string, such as "Intel", and a name also of type string, for example "Core i7-9850".

We need to keep track of how many cores or threads the CPU has. They cannot be negative, so let's use uint32 here.

Next, it has the minimum and maximum frequency, for example 2.4 Ghz or something like that. So we can use double type here.

Generate Go codes

Now we've finished our first protobuf message. How can we generate Go codes from it?

First, we need to install protocol buffer compiler (or protoc). On macOS, we can easily do that with the help of Homebrew.

You can install Homebrew with this simple command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Once Homebrew is installed, can run this command to install protoc:

brew install protobuf

We can check if it's working or not by running the protoc command.

Next we will go to grpc.io to copy and run 2 commands to install 2 libraries: the golang grpc library and the protoc-gen-go library.

go get -u google.golang.org/grpc
go get -u github.com/golang/protobuf/protoc-gen-go

Now we're all set! I will create a new folder named pb to store the generated Go codes.

pcbook
├── proto
│   └── processor_message.proto
├── pb
└── main.go

Then run this command to generate the codes:

protoc --proto_path=proto proto/*.proto --go_out=plugins=grpc:pb

Our proto file is located inside the proto folder, so we tell protoc to look for it in that folder.

With the go_out parameter, we tell protoc to use the grpc plugins to generate Go codes, and store them inside the pb folder that we've created before.

Now if we open that folder in vscode, we will see a new file processor_message.pb.go.

pcbook
├── proto
│   └── processor_message.proto
├── pb
│   └── processor_message.pb.go
└── main.go

Look inside, there's a CPU struct and all fields with the correct data types as we defined in our protocol buffer file.

const _ = proto.ProtoPackageIsVersion3

type CPU struct {
    Brand                string   `protobuf:"bytes,1,opt,name=brand,proto3" json:"brand,omitempty"`
    Name                 string   `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
    NumberCores          uint32   `protobuf:"varint,3,opt,name=number_cores,json=numberCores,proto3" json:"number_cores,omitempty"`
    NumberThreads        uint32   `protobuf:"varint,4,opt,name=number_threads,json=numberThreads,proto3" json:"number_threads,omitempty"`
    MinGhz               float64  `protobuf:"fixed64,5,opt,name=min_ghz,json=minGhz,proto3" json:"min_ghz,omitempty"`
    MaxGhz               float64  `protobuf:"fixed64,6,opt,name=max_ghz,json=maxGhz,proto3" json:"max_ghz,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (m *CPU) Reset()         { *m = CPU{} }
func (m *CPU) String() string { return proto.CompactTextString(m) }
func (*CPU) ProtoMessage()    {}
func (*CPU) Descriptor() ([]byte, []int) {
    return fileDescriptor_466578cecc6db379, []int{0}
}

func (m *CPU) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_CPU.Unmarshal(m, b)
}
func (m *CPU) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_CPU.Marshal(b, m, deterministic)
}
func (m *CPU) XXX_Merge(src proto.Message) {
    xxx_messageInfo_CPU.Merge(m, src)
}
func (m *CPU) XXX_Size() int {
    return xxx_messageInfo_CPU.Size(m)
}
func (m *CPU) XXX_DiscardUnknown() {
    xxx_messageInfo_CPU.DiscardUnknown(m)
}

var xxx_messageInfo_CPU proto.InternalMessageInfo

func (m *CPU) GetBrand() string {
    if m != nil {
        return m.Brand
    }
    return ""
}

func (m *CPU) GetName() string {
    if m != nil {
        return m.Name
    }
    return ""
}

func (m *CPU) GetNumberCores() uint32 {
    if m != nil {
        return m.NumberCores
    }
    return 0
}

func (m *CPU) GetNumberThreads() uint32 {
    if m != nil {
        return m.NumberThreads
    }
    return 0
}

func (m *CPU) GetMinGhz() float64 {
    if m != nil {
        return m.MinGhz
    }
    return 0
}

func (m *CPU) GetMaxGhz() float64 {
    if m != nil {
        return m.MaxGhz
    }
    return 0
}

There are some special fields used internally by gRPC to serialise the message, but we don't need to care about them. Some useful getter functions are also generated. So it looks great!

Write a Makefile

The command that we used to generate codes is pretty long, so it’s not very convenient to type when we update the proto file and want to regenerate the codes. So let's create a Makefile with a short and simple command to do that.

pcbook
├── proto
│   └── processor_message.proto
├── pb
│   └── processor_message.pb.go
├── main.go
└── Makefile

In this Makefile, we add a gen task to run code generation command, a clean task to remove all generated go files whenever we want, and a run task to run the main.go file as well.

gen:
    protoc --proto_path=proto proto/*.proto --go_out=plugins=grpc:pb

clean:
    rm pb/*.go 

run:
    go run main.go

We can try them in the terminal.

When we run make clean, the generated files will be deleted.

When we run make gen, the files will be regenerated in pb folder.

And finally, when we run make run, "Hello world" is printed.

What's next

OK, so now you know how to define a simple protocol buffer message and generate Go code from it. In the next lecture, we will dig deeper and learn more advanced features of protobuf.

Thanks for reading! Happy coding, and see you later!

If you like the article, please subscribe to our Youtube channel and follow us on Twitter for more tutorials in the future.

If you want to join me on my current amazing team at Voodoo, check out our job openings here. Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.