Implement unary gRPC API in Go

There are 4 types of gRPC: unary, client-streaming, server-streaming and bidirectional streaming. In this lecture, we will learn how to implement the simplest one: unary RPC.

Here's the link to the full gRPC course playlist on Youtube
Github repository: pcbook-go and pcbook-java
Gitlab repository: pcbook-go and pcbook-java

Define a proto service and a unary RPC

First step, we will create a new laptop_service.proto file.

In this file, we define a CreateLaptopRequest message, which contains only 1 field: the laptop we want to create.

message CreateLaptopRequest {
  Laptop laptop = 1;
}

Then the CreateLaptopResponse message also has only 1 field: the ID of the created laptop.

message CreateLaptopResponse {
  string id = 1;
}

We define the LaptopService with the keyword service. Then inside it, a unary RPC is defined as follow:

service LaptopService {
  rpc CreateLaptop(CreateLaptopRequest) returns (CreateLaptopResponse) {};
}

Start with the keyword rpc, then the name of the RPC is CreateLaptop. It takes a CreateLaptopRequest as input, and returns a CreateLaptopResponse. End it with a pair of curly brackets and a semicolon.

And that's it! Pretty easy and straight-forward.

Generate codes for the unary RPC

Now let's open the terminal and generate codes for this new RPC with this command:

make gen

The laptop_service.pb.go will be generated inside the pb folder.

Let's take a closer look at its content:

type CreateLaptopRequest struct {
    Laptop               *Laptop  `protobuf:"bytes,1,opt,name=laptop,proto3" json:"laptop,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (m *CreateLaptopRequest) GetLaptop() *Laptop {
    if m != nil {
        return m.Laptop
    }
    return nil
}

This is the CreateLaptopRequest struct. It has a function to get the input laptop object.

type CreateLaptopResponse struct {
    Id                   string   `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (m *CreateLaptopResponse) GetId() string {
    if m != nil {
        return m.Id
    }
    return ""
}

This is the CreateLaptopResponse struct. It has a function to get the output ID of the laptop.

// LaptopServiceClient is the client API for LaptopService service.
type LaptopServiceClient interface {
    CreateLaptop(ctx context.Context, in *CreateLaptopRequest, opts ...grpc.CallOption) (*CreateLaptopResponse, error)
}

This is the LaptopServiceClient interface. It has a function CreateLaptop, just like how we defined it in the proto file.

Why is it an interface? Because it will allow us to implement our own custom client, such as a mock client that can be used for unit testing.

type laptopServiceClient struct {
    cc *grpc.ClientConn
}

func NewLaptopServiceClient(cc *grpc.ClientConn) LaptopServiceClient {
    return &laptopServiceClient{cc}
}

func (c *laptopServiceClient) CreateLaptop(ctx context.Context, in *CreateLaptopRequest, opts ...grpc.CallOption) (*CreateLaptopResponse, error) {
    out := new(CreateLaptopResponse)
    err := c.cc.Invoke(ctx, "/techschool.pcbook.LaptopService/CreateLaptop", in, out, opts...)
    if err != nil {
        return nil, err
    }
    return out, nil
}

This laptopServiceClient struct with a small letter l Is the real implementation of the interface.

Next, is the LaptopServiceServer struct.

// LaptopServiceServer is the server API for LaptopService service.
type LaptopServiceServer interface {
    CreateLaptop(context.Context, *CreateLaptopRequest) (*CreateLaptopResponse, error)
}

It is also an interface with no implementation. Basically, we must write our own implementation of the server. It must have the CreateLaptop function as defined in this interface.

Here's the function to register the laptop service on a specific gRPC server so that it can receive and handle requests from client:

func RegisterLaptopServiceServer(s *grpc.Server, srv LaptopServiceServer) {
    s.RegisterService(&_LaptopService_serviceDesc, srv)
}

Implement the server's unary RPC handler

Now let's implement the LaptopServiceServer!

I will create a new service folder, and create a laptop_server.go file inside it.

I will declare a LaptopServer struct and a NewLaptopServer() function to return a new instance of it:

// LaptopServer is the server that provides laptop services
type LaptopServer struct {
}

// NewLaptopServer returns a new LaptopServer
func NewLaptopServer() *LaptopServer {
    return &LaptopServer{}
}

Now we need to implement the CreateLaptop function, which is required by the LaptopServiceServer interface.

It takes a context and a CreateLaptopRequest object as input, and returns a CreateLaptopResponse or an error.

// CreateLaptop is a unary RPC to create a new laptop
func (server *LaptopServer) CreateLaptop(
    ctx context.Context,
    req *pb.CreateLaptopRequest,
) (*pb.CreateLaptopResponse, error) {
    laptop := req.GetLaptop()
    log.Printf("receive a create-laptop request with id: %s", laptop.Id)
}

First we call GetLaptop function to get the laptop object from the request.

If the client has already generated the laptop ID, we must check if it is a valid UUID or not.

To do that, we use the Google UUID package. Run this command in the terminal to install it:

go get github.com/google/uuid

After that, we can use uuid.Parse() function to parse the laptop ID.

If it returns an error then it means the provided ID is invalid, we should return a nil response to the client together with an error status code.

For that, we use the status and codes subpackages of the grpc package. We return the InvalidArgument code in this case because the laptop ID is provided by the client.

if len(laptop.Id) > 0 {
    // check if it's a valid UUID
    _, err := uuid.Parse(laptop.Id)
    if err != nil {
        return nil, status.Errorf(codes.InvalidArgument, "laptop ID is not a valid UUID: %v", err)
    }
} else {
    id, err := uuid.NewRandom()
    if err != nil {
        return nil, status.Errorf(codes.Internal, "cannot generate a new laptop ID: %v", err)
    }
    laptop.Id = id.String()
}

If the client hasn't sent the laptop ID, we will generate it on the server with uuid.NewRandom() command.

If an error occurs, we return it with the codes.Internal, meaning an internal server error. Else, we just set the laptop.ID to the generated random UUID.

Next we should check if the request is timeout or cancelled by the client or not, because if it is then there's no reason to continue processing the request.

To check this, we simply use the ctx.Err() function:

if ctx.Err() == context.Canceled {
    log.Print("request is canceled")
    return nil, status.Error(codes.Canceled, "request is canceled")
}

if ctx.Err() == context.DeadlineExceeded {
    log.Print("deadline is exceeded")
    return nil, status.Error(codes.DeadlineExceeded, "deadline is exceeded")
}

Implement in-memory storage to save laptops

Normally after this, we should save the laptop to the database. However, this is a course about gRPC so I just want to focus on it. Therefore, to make it simple, I will just use an in-memory storage. It will also be very useful for unit testing later.

Let's create a new laptop_store.go file inside the service folder.

As we might have different types of store, I define LaptopStore as an interface. It has a Save() function to save a laptop to the store.

// LaptopStore is an interface to store laptop
type LaptopStore interface {
    // Save saves the laptop to the store
    Save(laptop *pb.Laptop) error
}

Then we will write an InMemoryLaptopStore to implement this interface. Later if we want to save laptops to the database, we can always implement another DBLaptopStore to do so.

// InMemoryLaptopStore stores laptop in memory
type InMemoryLaptopStore struct {
    mutex sync.RWMutex
    data  map[string]*pb.Laptop
}

In this InMemoryLaptopStore, we use a map to store the data, where the key is the laptop ID, and the value is the laptop object.

We need a read-write mutex to handle the multiple concurrent requests to save laptops.

Now let's declare a function to return a new InMemoryLaptopStore, and initialise the data map inside it:

// NewInMemoryLaptopStore returns a new InMemoryLaptopStore
func NewInMemoryLaptopStore() *InMemoryLaptopStore {
    return &InMemoryLaptopStore{
        data: make(map[string]*pb.Laptop),
    }
}

Then implement the Save laptop function as required by the interface.

First we need to acquire a write lock before adding new objects. Remember to defer the unlock command.

Next, we check if the laptop ID already exists in the map or not. If it does, just return an error to the caller.

Ther ErrAlreadyExists variable should be exported, so that it can be used from outside of this service package.

// ErrAlreadyExists is returned when a record with the same ID already exists in the store
var ErrAlreadyExists = errors.New("record already exists")

// Save saves the laptop to the store
func (store *InMemoryLaptopStore) Save(laptop *pb.Laptop) error {
    store.mutex.Lock()
    defer store.mutex.Unlock()

    if store.data[laptop.Id] != nil {
        return ErrAlreadyExists
    }

    other, err := deepCopy(laptop)
    if err != nil {
        return err
    }

    store.data[other.Id] = other
    return nil
}

If the laptop doesn't exist, we can save it to the store. However, to be safe, we should do a deep-copy of the laptop object.

For that, we can use the copier package:

go get github.com/jinzhu/copier

Then we can use it to implement the deepCopy() function:

func deepCopy(laptop *pb.Laptop) (*pb.Laptop, error) {
    other := &pb.Laptop{}

    err := copier.Copy(other, laptop)
    if err != nil {
        return nil, fmt.Errorf("cannot copy laptop data: %w", err)
    }

    return other, nil
}

Alright, let's go back to our laptop server and add a new laptopStore field to the LaptopServer struct.

// LaptopServer is the server that provides laptop services
type LaptopServer struct {
    laptopStore LaptopStore
}

// NewLaptopServer returns a new LaptopServer
func NewLaptopServer(laptopStore LaptopStore) *LaptopServer {
    return &LaptopServer{laptopStore}
}

Then in the CreateLaptop() function, we can call server.Store.Save() to save the input laptop to the store.

If there's an error, return codes.Internal with the error to the client. We can make it clearer to the client to handle, by checking if the error is already-exists-record or not.

To do that, we simply call errors.Is() function. If it's true, we return AlreadyExists status code instead of Internal.

func (server *LaptopServer) CreateLaptop(
    ctx context.Context,
    req *pb.CreateLaptopRequest,
) (*pb.CreateLaptopResponse, error) {
    ...

    err := server.laptopStore.Save(laptop)
    if err != nil {
        code := codes.Internal
        if errors.Is(err, ErrAlreadyExists) {
            code = codes.AlreadyExists
        }

        return nil, status.Errorf(code, "cannot save laptop to the store: %v", err)
    }

    log.Printf("saved laptop with id: %s", laptop.Id)

    res := &pb.CreateLaptopResponse{
        Id: laptop.Id,
    }
    return res, nil
}

Finally, if no errors occur, we can create a new response object with the laptop ID and return it to the caller. And that's it for the server.

Test the unary RPC handler

Now I'm gonna show you how to test it. Let's create a service/laptop_server_test.go file and set the package name to service_test. Then we create a function TestServerCreateLaptop().

I want to test many different cases, so let's use table-driven tests. A test case will have a name, an input laptop object, a laptop store, and an expected status code.

The 1st case is a successful call with laptop ID generated by the client. So the laptop will be a sample.NewLaptop(), store is just a new InMemoryLaptopStore, And the expected code is OK.

func TestServerCreateLaptop(t *testing.T) {
    t.Parallel()

    testCases := []struct {
        name   string
        laptop *pb.Laptop
        store  service.LaptopStore
        code   codes.Code
    }{
        {
            name:   "success_with_id",
            laptop: sample.NewLaptop(),
            store:  service.NewInMemoryLaptopStore(),
            code:   codes.OK,
        },
        {
            name:   "success_no_id",
            laptop: laptopNoID,
            store:  service.NewInMemoryLaptopStore(),
            code:   codes.OK,
        },
        {
            name:   "failure_invalid_id",
            laptop: laptopInvalidID,
            store:  service.NewInMemoryLaptopStore(),
            code:   codes.InvalidArgument,
        },
        {
            name:   "failure_duplicate_id",
            laptop: laptopDuplicateID,
            store:  storeDuplicateID,
            code:   codes.AlreadyExists,
        },
    }
}

The 2nd case is also a successful call, but with no laptop ID. I expect the server to generate a random ID for us. So here we generate a sample laptop, and set its ID to empty string.

laptopNoID := sample.NewLaptop()
laptopNoID.Id = ""

The 3rd case is a failed call because of an invalid UUID. So we generate a sample laptop and set its ID to invalid-uuid. And for this case, we expect the status code to be InvalidArgument.

laptopInvalidID := sample.NewLaptop()
laptopInvalidID.Id = "invalid-uuid"

The last case is a failed call because of duplicate ID. So we have to to create a laptop and save it to the store beforehand. We expect to see an AlreadyExists status code in this case.

laptopDuplicateID := sample.NewLaptop()
storeDuplicateID := service.NewInMemoryLaptopStore()
err := storeDuplicateID.Save(laptopDuplicateID)
require.Nil(t, err)

Alright, all test cases are ready. Now we iterate through them with a simple for loop.

We must save the current test case to a local variable. This is very important to avoid concurrency issues, because we want to create multiple parallel subtests.

To create a subtest, we call t.Run() and use tc.name for the name of the subtest. We call t.Parallel() to make it run in parallel with other tests.

for i := range testCases {
    tc := testCases[i]

    t.Run(tc.name, func(t *testing.T) {
        t.Parallel()

        req := &pb.CreateLaptopRequest{
            Laptop: tc.laptop,
        }

        server := service.NewLaptopServer(tc.store)
        res, err := server.CreateLaptop(context.Background(), req)
        ...
    })
}

Then we build a new CreateLaptopRequest object with the input tc.laptop. We create a new LaptopServer with the in-memory laptop store.

Then just call server.CreateLaptop() function with a background context and the request object.

Now there are 2 cases:

The successful case, or when tc.code is OK. In this case, we should check there's no error. The response should be not nil. The returned ID should be not empty. And if the input laptop already has ID, then the returned ID should equal to it.

...

res, err := server.CreateLaptop(context.Background(), req)

if tc.code == codes.OK {
    require.NoError(t, err)
    require.NotNil(t, res)
    require.NotEmpty(t, res.Id)
    if len(tc.laptop.Id) > 0 {
        require.Equal(t, tc.laptop.Id, res.Id)
    }
} else {
    require.Error(t, err)
    require.Nil(t, res)
    st, ok := status.FromError(err)
    require.True(t, ok)
    require.Equal(t, tc.code, st.Code())
}

The failure case, when tc.code is not OK. We check there should be an error and the response should be nil.

To check the status code, we call status.FromError() to get the status object. Check that ok should be true and st.Code() should equal to tc.code. Then it's done.

However, the tests that we’ve written didn't use any kind of network call yet. They're basically just a direct call on server side.

Test the unary RPC with the real connection

Now I will show you how to test the RPC request from the client side with a real connection.

Let's create laptop_client_test.go file. The package name is still service_test, but the function name is now TestClientCreateLaptop().

First we need to write a function to start the gRPC server. It will take a testing.T as an argument, and return the network address string of the server.

In this function, we create a new laptop server with an in-memory laptop store. We create the gRPC server by calling grpc.NewServer() function, then register the laptop service server on that gRPC server.

func startTestLaptopServer(t *testing.T, laptopStore service.LaptopStore) string {
    laptopServer := service.NewLaptopServer(laptopStore)

    grpcServer := grpc.NewServer()
    pb.RegisterLaptopServiceServer(grpcServer, laptopServer)

    listener, err := net.Listen("tcp", ":0") // random available port
    require.NoError(t, err)

    go grpcServer.Serve(listener)

    return listener.Addr().String()
}

We create a new listener that will listen to tcp connection. The number 0 here means that we want it to be assigned any random available port.

Then we just call grpcServer.Serve() to start listening to the request. This is a blocking call, so we have to run it in a separate go-routine, since we want to send requests to this server after that.

Finally we just return the the address string of the listener.

Next we will create another function to return a new laptop-client. This function will take the testing.T object, and the server address as its arguments, then return a pb.LaptopServiceClient.

func newTestLaptopClient(t *testing.T, serverAddress string) pb.LaptopServiceClient {
    conn, err := grpc.Dial(serverAddress, grpc.WithInsecure())
    require.NoError(t, err)
    return pb.NewLaptopServiceClient(conn)
}

First we dial the server address with grpc.Dial(). Since this is just for testing, we use an insecure connection.

We check that there is no error and return a new laptop service client with the created connection.

Now we can use 2 functions above to write the unit test:

func TestClientCreateLaptop(t *testing.T) {
    t.Parallel()

    laptopStore := service.NewInMemoryLaptopStore()
    serverAddress := startTestLaptopServer(t, laptopStore)
    laptopClient := newTestLaptopClient(t, serverAddress)

    laptop := sample.NewLaptop()
    expectedID := laptop.Id
    req := &pb.CreateLaptopRequest{
        Laptop: laptop,
    }

    ...
}

Here we create a new sample laptop, save its ID to a variable to compare later. We create a new request object with the laptop.

Then we use the laptopClient object to call CreateLaptop() function. We check that no error is returned and the response should be not nil. The returned ID should also match the expected ID we saved before.

func TestClientCreateLaptop(t *testing.T) {
    ...
    res, err := laptopClient.CreateLaptop(context.Background(), req)
    require.NoError(t, err)
    require.NotNil(t, res)
    require.Equal(t, expectedID, res.Id)
}

Now we want to make sure that the laptop is really stored on the server. To do that, we need to add 1 more function to the laptop store.

It's the Find() function to search for a laptop by its ID. It takes a string ID as an input and returns a laptop object with an error.

type LaptopStore interface {
    // Save saves the laptop to the store
    Save(laptop *pb.Laptop) error
    // Find finds a laptop by ID
    Find(id string) (*pb.Laptop, error)
}

// Find finds a laptop by ID
func (store *InMemoryLaptopStore) Find(id string) (*pb.Laptop, error) {
    store.mutex.RLock()
    defer store.mutex.RUnlock()

    laptop := store.data[id]
    if laptop == nil {
        return nil, nil
    }

    return deepCopy(laptop)
}

In this function, we first call mutex.RLock() to acquire a read lock.

Then we get the laptop from the store.data map by its id. If it's not found, just return nil. Else, we should return a deep-copy of the found laptop.

Now go back to our client test. We call laptopServer.Store.Find() to find laptop by ID. Check there's no error, and the laptop should be not nil.

func TestClientCreateLaptop(t *testing.T) {
    ...

    // check that the laptop is saved to the store
    other, err := laptopStore.Find(res.Id)
    require.NoError(t, err)
    require.NotNil(t, other)

    // check that the saved laptop is the same as the one we send
    requireSameLaptop(t, laptop, other)
}

Finally, we want to check that the saved laptop is the same as the one we sent. Let's write a separate function for that.

It will have 3 inputs: the testing.T object, and 2 laptop objects.

Now if we just use require.Equal() function to compare these 2 objects, the test will fail.

It’s because in the Laptop struct, there are some special fields that are used internally by gRPC to serialise the objects. Therefore, to compare 2 laptops properly, we must ignore those special fields.

One easy way is to just serialise the objects to JSON, and compare the 2 output JSON strings:

func requireSameLaptop(t *testing.T, laptop1 *pb.Laptop, laptop2 *pb.Laptop) {
    json1, err := serializer.ProtobufToJSON(laptop1)
    require.NoError(t, err)

    json2, err := serializer.ProtobufToJSON(laptop2)
    require.NoError(t, err)

    require.Equal(t, json1, json2)
}

Write the main server and client

Next, we will implement the main.go entry point for the gRPC server and client.

Let's create a new cmd folder, then in this folder, create 1 folder for the server and 1 folder for the client. Each will have its own main.go file.

Then also update the Makefile to have 2 run commands for the server and client binaries:

server:
    go run cmd/server/main.go -port 8080

client:
    go run cmd/client/main.go -address 0.0.0.0:8080

Now let's implement the server/main.go.

We need a port for the server, so I use the flag.Int() function to get it from command line arguments.

Similar to what we wrote in the unit test, we create a new laptop server object with an in-memory store. Then we create a new gRPC server and register the laptop server with it.

func main() {
    port := flag.Int("port", 0, "the server port")
    flag.Parse()
    log.Printf("start server on port %d", *port)

    laptopStore := service.NewInMemoryLaptopStore()
    laptopServer := service.NewLaptopServer(laptopStore)
    pb.RegisterLaptopServiceServer(grpcServer, laptopServer)

    address := fmt.Sprintf("0.0.0.0:%d", *port)
    listener, err := net.Listen("tcp", address)
    if err != nil {
        log.Fatal("cannot start server: ", err)
    }

    err = grpcServer.Serve(listener)
    if err != nil {
        log.Fatal("cannot start server: ", err)
    }
}

We create an address string with the port we get before, then listen for TCP connections on this server address.

Finally we call grpcServer.Serve() to start the server. If any error occurs, just write a fatal log and exit. And that's the server code.

Now the client. First we get the server address from the command line arguments.

We call grpc.Dial() function with the input address, and just create an insecure connection for now.

If an error occurs, we write a fatal log and exit. Else, we create a new laptop client object with the connection.

func main() {
    serverAddress := flag.String("address", "", "the server address")
    flag.Parse()
    log.Printf("dial server %s", *serverAddress)

    conn, err := grpc.Dial(*serverAddress, grpc.WithInsecure())
    if err != nil {
        log.Fatal("cannot dial server: ", err)
    }

    laptopClient := pb.NewLaptopServiceClient(conn)

    laptop := sample.NewLaptop()
    req := &pb.CreateLaptopRequest{
        Laptop: laptop,
    }

    // set timeout
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    res, err := laptopClient.CreateLaptop(ctx, req)
    if err != nil {
        st, ok := status.FromError(err)
        if ok && st.Code() == codes.AlreadyExists {
            // not a big deal
            log.Print("laptop already exists")
        } else {
            log.Fatal("cannot create laptop: ", err)
        }
        return
    }

    log.Printf("created laptop with id: %s", res.Id)
}

Then we generate a new laptop, make a new request object, and just call laptopClient.Createlaptop() function with the request and a context. Here we use context.WithTimeout() to set the timeout of this request to be 5 seconds.

If error is not nil, we convert it into a status object. If the status code is AlreadyExists then it’s not a big deal, just write a normal log. Else, we write a fatal log.

If everything is good, we simply write a log saying the laptop is created with this ID. And that's it for the client.

So now you know how to implement and test a unary gRPC request with Go. Thanks for reading! Happy coding and I'll catch you guys in the next lecture.

If you like the article, please subscribe to our Youtube channel and follow us on Twitter for more tutorials in the future.

If you want to join me on my current amazing team at Voodoo, check out our job openings here. Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.