Build and Deploy: With AWS (Lambda, API Gateway and DynamoDB) using Golang

As of 2024, AWS has officially deprecated the go1.x runtime which came into effect on December 31,2023 as mentioned in their official blog post. In this post, let's revisit and look at how you can build a REST API that interacts with DynamoDB and deploy it as a lambda function using Amazon Linux 2023 runtime.

For demonstration, I have only shown two functionalities (fetch and create) in this article. But the complete code with all the functionalities can be found here.

There are a few standard ways you can deploy the code as a lambda function, such as using a container(Docker), using serverless framework, using terraform scripts or using AWS SAM.
But using .zip file archives to deploy into AWS lambda is one of the more straight forward ways, as mentioned in the official AWS documentation.

Table of contents

1. Prerequisites

2. WRITE the code for the Golang REST API

3. BUILD the executable file

4. DEPLOY to AWS

5. TEST the API

1. Prerequisites

1.1. Golang should be installed in your system.

To check wether GO is installed in your system or not use the command go version in your terminal. If it is not installed, check the official installation steps.

1.2. You should have an AWS account and preferrably logged in as an IAM user.

To sign in as a root or an IAM user follow the steps mentioned in the official documentation.

1.3. Use a code editor of your choice.

1.4. AWS CLI should be installed and configured.

Creating policies and roles as well as deploying the lambda function can be done using the AWS management console as well as the AWS CLI(on the terminal). In case of using AWS CLI, it should be properly configured.

2. WRITE the code for the Golang REST API

2.1. Folder Structure

└── 📁player-api
    └── 📁cmd
        └── main.go
    └── go.mod
    └── go.sum
    └── 📁pkg
        └── 📁handlers
            └── handlers.go
            └── response.go
        └── 📁player
            └── player.go

NOTE

You may structure the project folder anyway you want. But it is always a good practice to follow a standard pattern to organize the folder. The above structure is one of the ways you can set up your Go projects.

2.2 Create your GO Project

Before writing your code you need to setup the directory where you will house your project. After that, open the terminal from the directory, and enter the following command to initialize your project.

# go mod init [module path]
go mod init github.com/Sourjaya/player-api

The go mod init command creates a go.mod file to track your code's dependencies. Using your own github repository will provide a unique module path for the project.
Now, in main.go write the following code:

In main.go create a session by passing in the region, then create an instance to the DynamoDB client and use lambda.Start(handler) to interact with an internal Lambda endpoint to pass requests to the handler. Then in the handler function check for the HTTP method type of the request and call the appropriate functions and return those functions.
Make sure you have an environment variable named AWS_REGION in your system. To create the environment variable in Linux, use this command:

#Open ~/.profile
sudo nano ~/.profile

Then write the export command at the end of the file:

#export AWS_REGION=[your_aws_region]
export AWS_REGION=ap-south-1

And then press CTRL+S to save and CTRL+X to exit nano. Now logoutof your system and log back in.

After you have written the main file, in pkg/handlers/handlers.go define and write the functions for each http method type. Write the following code:

The GetPlayer function calls GetPlayerByID if ID is provided as URL parameter, to fetch the player information with respect to the provided key, else it calls GetPlayers to fetch information of all the players in the table. The two functions are available in the players package.

The CreatePlayer function calls CreatePlayer from the player package to add a new player record in the table. Both GetPlayer and Create Player takes request, tablename and client as function parameters and returns a response and a http status code. The Unhandled function return a 405 status code and a error message.

The response method is responsible to create the APIGatewayProxyResponse which will contain appropriate headers, status code and the json response body.

Now, in player.go write the functions that will interact with DynamoDB.

Function GetPlayerByID takes the ID, tablename and client as parameters and returns a Player structure containing the player info and an error(if found).

Function GetPlayers takes tablename and client as parameters and returns a slice(list) containing info about each player in the database table along with an error(if found).

Function Create Player takes request, tablename and client as parameters and returns a Player structure containing the player info and an error(if found).

3. BUILD the executable file

3.1 Install the dependencies

Now that you have written all the code needed to build the executable file, it is necessary to install all the dependencies and remove unused modules. To do that, in the terminal execute the command:

# This will generate go.sum file
go mod tidy

3.2 Create the executable file and zip it

If you are using Linux/Mac system execute this command(according to your system architecture use any one of the following commands) to generate the executable:

For x86_64 architecture

GOOS=linux GOARCH=amd64 go build -tags lambda.norpc -o bootstrap main.go

For arm64 architecture

GOOS=linux GOARCH=arm64 go build -tags lambda.norpc -o bootstrap main.go

(Optional) To create a stable binary package for standard C library (libc) versions

GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -o bootstrap -tags lambda.norpc main.go

If you are using Windows system execute the following set of commands in terminal(using CGO_ENABLED is optional):

For x86_64 architecture

set GOOS=linux
set GOARCH=amd64
go build -tags lambda.norpc -o bootstrap main.go

For arm64 architecture

set GOOS=linux
set GOARCH=arm64
go build -tags lambda.norpc -o bootstrap main.go

NOTE

Since we are using provided.al2023(Amazon Linux 2023 runtime) we have to name our go executable binary file as bootstrap.

Now to create the .zip file in Linux/Mac:

# zip -jrm [zip file name along with the extension]
# the -m flag deletes the executable 
# file which is not needed anymore
zip -jrm football_api.zip bootstrap

And to create the .zip file in Windows :

tar -acf football_api.zip bootstrap
del bootstrap

4. DEPLOY to AWS

4.1 Create the Lambda Function

4.1.1 Using AWS CLI

Create Execution Policy

To create an execution policy create a json file with the following content:
policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1428341300017",
            "Action": [
                "dynamodb:DeleteItem",
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:UpdateItem"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Sid": "",
            "Resource": "*",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Effect": "Allow"
        }
    ]
}

Now execute the command:

#Syntax
:'
aws iam create-policy \
    --policy-name [policy_name] \
    --policy-document file://[file_path/file_name]
'

#Command
aws iam create-policy \
    --policy-name lambda-api \
    --policy-document file://./policy.json

You have successfully created a policy. Remember to note the policy ARN from the response in the terminal. The policy ARN would look something like arn:aws:iam::111111111111:policy/lambda-api

Create Execution Role and attach Policy

To create an execution role create a json file with the following content:
trust-policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Now execute the command:

#Syntax
:'
aws iam create-role --role-name [role_name] \
--assume-role-policy-document file://[file_path/file_name]
'

#Command
#Remember to note the role ARN as well.
aws iam create-role --role-name lambda-api \
--assume-role-policy-document file://./trust-policy.json

Now, we have to attach the policy we created to this role.
To do that use this command:

#Syntax
:'
aws iam attach-role-policy --role-name [role_name] \
--policy-arn [policy_arn that was noted]
'
#Command
aws iam attach-role-policy --role-name lambda-api \
--policy-arn arn:aws:iam::111111111111:policy/lambda-api

Create Lambda function

To create a lambda function using AWS CLI use the following command in the terminal:

# Syntax
:'
aws lambda create-function --function-name [lambda_function_name] \
--runtime provided.al2023 --handler [handler_name] \
--architectures [your_system_architecture] \
--role arn:aws:iam::[aws_id]:role/[role_name] \
--zip-file fileb://[zip_file]
'

# Command
aws lambda create-function --function-name new-lambda \
--runtime provided.al2023 --handler bootstrap \
--role arn:aws:iam::111111111111:role/lambda-api \
--zip-file fileb://football_api.zip

NOTE

• The --architectures option is only required if you're using arm64. The default value is x86_64.
• For --role, specify the Amazon Resource Name (ARN) of the execution role.
• I have not shown my AWS ID in any of the above commands. Put your 12-digit AWS ID in place of those 1's and use it.
• Replace '\' from the commands with '^' if you are using a Windows system

4.1.2 Using AWS Management Console

Login to the management console. And follow the steps.

Create Execution Policy

To create an execution policy:

1. Select Security credentials from the dropdown when you click on your account at the top right.

1. Go to Access management-->Policies .
2. Click on Create Policy button.
3. Now in Policy editor and under JSON tab put the content of policy.json file.
4. Now click on Next and give it a Name(lambda-api) and Description(optional) and then click Create Policy.

You have successfully created a policy.

Create Execution Role and attach Policy

To create an Execution Role:

1. Go to Access management-->Roles .
2. Under Trusted entity type select AWS service and under Use case select lambda and click Next.
3. In Add permissions search and select the policy(lamda-api) you created and click Next.
4. Now give it a Name(lambda-api) and click Create role.

Create Lambda function

To create a lambda function using AWS Console follow the steps:

1. Search Lambda in the search box and go to the Functions page.
2. Click on Create Function button.
3. Give the function a Name(new-lambda), select the Runtime(Amazon Linux 2023) and select the Architecture(x86_64 in my case).
4. Under Permissions tab change the execution role.
5. You may choose the role that was created from Use an existing role or choose Simple microservice permissions from Policy templates by selecting Create a new role from AWS policy templates .
6. Now after you click on Create function you have to Edit the Runtime Settings and change handler name to bootstrap.
7. The only step that is left is to upload the Code Source which will be the zip file that was created.

You have successfully created a Lambda function.

4.2 Create a table in DynamoDB

We can create a table in DynamoDB using both the AWS CLI or the Console. I prefer to use the Console, as the table attributes are created through code. But if you want to learn how to use AWS CLI to use DynamoDB follow this link.

To create a table using DynamoDB:

1. Search DynamoDB in the search box and go to the Tables page.
2. Click on Create Table and give the table a Name(you have mentioned in your code).
3. Enter Partition Key(which will be the primary key). In this case it will be the id field.
4. Click on Create Table.

4.3 Create a REST API in APIGateway

To create the API:

1. Search API Gateway in the search box and in the API page click on Create API.
2. Now scroll down and click Build on REST API tab.
3. Under API details tab select New API and then give tha API a name and select Regional in API endpoint type.
4. After you click on Create API, the page will be redirected to Resources under the API. Click on Create method under Methods tab.
5. Select ANY as Method type, Chose the Lambda function name(or alias) and the region(in my case it was ap-south-1) .
6. Enable the Lambda proxy integration and Default timeout options and click on Create method.
7. The final step is to click on Deploy API, create a New Stage(staging) and Deploy. Make sure to note the Invoke URL which will look like this: https://unp18fmgcc.execute-api.ap-south-1.amazonaws.com/staging.

You will see something like this:

You have successfully deployed a REST API using a serverless stack. Now we will look at how we can Test the API and wether the endpoints are working or not.

5. TEST the API

The API will have the following Endpoints:

You can Test your API using the good old curl command. The commands are given below:

1. Get All Players.

#Syntax
#curl -X GET [Invoke URL]
#Command
curl -X GET https://unp18fmgcc.execute-api.ap-south-1.amazonaws.com/staging

2. Get Player by ID.

#Syntax
#curl -X GET [Invoke URL]?id=[id]
#Command
curl -X GET https://unp18fmgcc.execute-api.ap-south-1.amazonaws.com/staging?id=7ff4d720-a825-4727-ad95-e37dd62b90cf

3. Create Player.

#Syntax
:'
curl --header "Content-Type: application/json" --request POST --data [JSON data] [Invoke URL]
'
#Command
curl --header "Content-Type: application/json" --request POST --data '{"firstName": "Luis", "lastName": "Suarez",    "country":"uru","position":"FW","club":"Inter Miami"}' https://unp18fmgcc.execute-api.ap-south-1.amazonaws.com/staging

Postman can also be used to Test the endpoints and you can make a postman collection similar to:

References

1. AWS Lambda
2. AWS DynamoDB
3. AWS APIGateway
4. AWS Tutorials

I wanted to document what I learned and how I deployed to AWS lambda in an article so that beginners like me can get help from it.

If you found this article informative, and if you have any feedback, do drop a comment. And let's connect on 𝕏. Thank You.