DEV Community

Cover image for Local Serverless Application Development with AWS SAM and Localstack
AWS Community Builders profile image Jimmy
Jimmy for AWS Community Builders

Posted on • Updated on

Local Serverless Application Development with AWS SAM and Localstack

Are you looking for cost-efficient and streamlined local development and testing of serverless applications built with AWS services like Amazon API Gateway, AWS Lambda, and Amazon DynamoDB?

With AWS SAM and Localstack, you can emulate AWS services on a local machine and allow developers to work offline, saving costs and enabling rapid iteration.

In this article, we will explore on how to build a serverless API service and test it locally with AWS SAM and Localstack.

Follow me for more

FullStack Saiyan | Twitter, Instagram, Facebook | Linktree

Share about Web, Mobile, Backend, Frontend, and Cloud Computing.

favicon linktr.ee

AWS Serverless Application Model (SAM)

aws-sam
AWS SAM (Serverless Application Model) is an open-source framework for building serverless applications. It extends AWS CloudFormation to provide a simplified way of defining the Amazon API Gateway APIs, AWS Lambda functions, and Amazon DynamoDB tables needed by your serverless application.

To install AWS SAM you can refer to this AWS documentation page for detailed installation. Install AWS SAM

We will also need to install AWS CLI. You can refer to this documentation Install AWS CLI.

Localstack

localstack
Localstack is an open-source project that provides a fully functional local AWS cloud stack for development and testing purposes. It allows developers to emulate several AWS services on their local machines, enabling them to develop and test their applications without accessing the actual AWS cloud.

LocalStack aims to replicate the behavior of various AWS services, such as Amazon S3, Amazon DynamoDB, AWS Lambda, Amazon SQS, and many others. Developers can interact with these emulated services through the AWS SDKs or command-line tools, just as they would with the real AWS services.

To install Localstack you can refer to this AWS documentation page for detailed installation. Install Localstack

Docker

Both AWS SAM and Localstack utilize Docker to run simulate AWS Service locally, so we will be using Docker also.

docker

Docker is a platform and a set of tools designed to simplify the creation, deployment, and running of applications using containers. Containers are lightweight, portable, and consistent environments that encapsulate an application and its dependencies. Docker allows developers to package an application and its dependencies into a container image, which can then be easily distributed and run consistently across different environments.

To install Docker, you can also check Docker documentation page for detailed installation. Install Docker Engine

Create project with AWS SAM.

  • We can start creating and initializing our AWS SAM project using this command. 
sam init
Enter fullscreen mode Exit fullscreen mode
  • Then we can follow this configuration wizard to start creating our Todo API service. --- For the template source we can choose 1- AWS Quick Start Templates 
You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
Enter fullscreen mode Exit fullscreen mode
  • For the AWS Quick Start application template, we can choose 7 - Serverless API 
Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Data processing
        3 - Hello World Example with Powertools for AWS Lambda
        4 - Multi-step workflow
        5 - Scheduled task
        6 - Standalone function
        7 - Serverless API
        8 - Infrastructure event management
        9 - Lambda Response Streaming
        10 - Serverless Connector Hello World Example
        11 - Multi-step workflow with Connectors
        12 - GraphQLApi Hello World Example
        13 - Full Stack
        14 - Lambda EFS example
        15 - Hello World Example With Powertools for AWS Lambda
        16 - DynamoDB Example
        17 - Machine Learning
Template: 7
Enter fullscreen mode Exit fullscreen mode
  • For runtime we can choose 2 - nodejs18.x 
Which runtime would you like to use?
        1 - dotnet6
        2 - nodejs18.x
        3 - nodejs16.x
        4 - nodejs14.x
Runtime: 2
Enter fullscreen mode Exit fullscreen mode
  • For enable X-Ray tracing, choose N 
Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N
Enter fullscreen mode Exit fullscreen mode
  • Choose N for enable monitoring using CloudWatch Application Insights 
Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: N
Enter fullscreen mode Exit fullscreen mode
  • Project name, type serverless-api 
Project name [sam-app]: serverless-api
Enter fullscreen mode Exit fullscreen mode
  • The complete SAM CLI configuration will look like this. 
You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Data processing
        3 - Hello World Example with Powertools for AWS Lambda
        4 - Multi-step workflow
        5 - Scheduled task
        6 - Standalone function
        7 - Serverless API
        8 - Infrastructure event management
        9 - Lambda Response Streaming
        10 - Serverless Connector Hello World Example
        11 - Multi-step workflow with Connectors
        12 - GraphQLApi Hello World Example
        13 - Full Stack
        14 - Lambda EFS example
        15 - Hello World Example With Powertools for AWS Lambda
        16 - DynamoDB Example
        17 - Machine Learning
Template: 7

Which runtime would you like to use?
        1 - dotnet6
        2 - nodejs18.x
        3 - nodejs16.x
        4 - nodejs14.x
Runtime: 2

Based on your selections, the only Package type available is Zip.
We will proceed to selecting the Package type as Zip.

Based on your selections, the only dependency manager available is npm.
We will proceed copying the template using npm.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: N

Project name [sam-app]: serverless-api

    -----------------------
    Generating application:
    -----------------------
    Name: serverless-api
    Runtime: nodejs18.x
    Architectures: x86_64
    Dependency Manager: npm
    Application Template: quick-start-web
    Output Directory: .
    Configuration file: serverless-api\samconfig.toml

    Next steps can be found in the README file at serverless-api\README.md


Commands you can use next
=========================
[*] Create pipeline: cd serverless-api && sam pipeline init --bootstrap
[*] Validate SAM template: cd serverless-api && sam validate
[*] Test Function in the Cloud: cd serverless-api && sam sync --stack-name {stack-name} --watch
Enter fullscreen mode Exit fullscreen mode

AWS SAM will then generate our serverless-api project according to the configuration we chose above like this.

.
|-- README.md
|-- __tests__
|   `-- unit
|       `-- handlers
|-- buildspec.yml
|-- env.json
|-- events
|   |-- event-get-all-items.json
|   |-- event-get-by-id.json
|   `-- event-post-item.json
|-- package.json
|-- samconfig.toml
|-- src
|   `-- handlers
|       |-- get-all-items.mjs
|       |-- get-by-id.mjs
|       `-- put-item.mjs
`-- template.yaml
Enter fullscreen mode Exit fullscreen mode

src/handlers is where our Lambda function will be, and template.yaml is our AWS SAM template.

Our project architecture above will look like this

architecture

In this project, we are using several AWS Services like

  • API Gateway
  • Lambda
  • DynamoDB

So we need a way to emulate these services locally.

Next, we will explore how to run our project and emulate AWS services locally using AWS SAM.

Run and Test Locally with AWS SAM

Run DynamoDB locally

AWS SAM doesn't emulate DynamoDB service that we are going to use in our project. So we need to run DynamoDB locally.

Amazon DynamoDB provides a way to run DynamoDB locally on our computer. Deploying DynamoDB locally on your computer

So we will run our DynamoDB locally using Docker.

  • Create docker-compose.yml
  • Type the following code 
version: '3.8'
services:
 dynamodb-local:
   command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
   image: "amazon/dynamodb-local:latest"
   container_name: dynamodb-local
   ports:
     - "8000:8000"
   volumes:
     - "./docker/dynamodb:/home/dynamodblocal/data"
   working_dir: /home/dynamodblocal
Enter fullscreen mode Exit fullscreen mode
  • Run the following command in your terminal 
docker-compose up
Enter fullscreen mode Exit fullscreen mode
  • Check whether our Dynamodb container is successfully running 
$ docker ps
CONTAINER ID   IMAGE                                          COMMAND                  CREATED          STATUS          PORTS                      NAMES
e7d0b58896e5   amazon/dynamodb-local:latest                   "java -jar DynamoDBL…"   2 months ago     Up 2 hours      0.0.0.0:8000->8000/tcp     dynamodb-local
Enter fullscreen mode Exit fullscreen mode

Take note on our dynamodb container name which is dynamodb-local and the dynamodb network named dynamodb_default. We will be using this information to configure our application to run locally depending on the ENVIRONMENT variable.

Create our DynamoDB table inside our local DynamoDB

To create our DynamoDB table inside our local DynamoDB, we can run this command

aws dynamodb create-table \
    --table-name SampleTable \
    --attribute-definitions \
        AttributeName=id,AttributeType=S \
    --key-schema \
        AttributeName=id,KeyType=HASH \
    --provisioned-throughput \
        ReadCapacityUnits=2,WriteCapacityUnits=2 \
    --table-class STANDARD \
    --endpoint-url http://localhost:8000
Enter fullscreen mode Exit fullscreen mode

As you notice, we will run above command into our local DynamoDB endpoint using this flag --endpoint-url HTTP://localhost:8000

Modify our template.yaml and function handlers

  • First we will modify our template.yaml by adding this configuration 
Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "local"
Enter fullscreen mode Exit fullscreen mode

So that our complete template.yaml configuration will looks like this.

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  serverless-api

Transform:
  - AWS::Serverless-2016-10-31

Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "local"

Resources:
  getAllItemsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-all-items.getAllItemsHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP get method to get all items from a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /
            Method: GET

  getByIdFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-by-id.getByIdHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP get method to get one item by id from a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /{id}
            Method: GET

  putItemFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/put-item.putItemHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A simple example includes a HTTP post method to add one item to a DynamoDB table.
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Environment:
        Variables:
          SAMPLE_TABLE: !Ref SampleTable
      Events:
        Api:
          Type: Api
          Properties:
            Path: /
            Method: POST

  SampleTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      PrimaryKey:
        Name: id
        Type: String
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

Outputs:
  WebEndpoint:
    Description: "API Gateway endpoint URL for Prod stage"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
Enter fullscreen mode Exit fullscreen mode
  • Next, we will modify all our function handler get-all-items.mjs, get-by-id.mjs and put-item-mjs. We will modify this part of the code. 
const client = new DynamoDBClient();
Enter fullscreen mode Exit fullscreen mode

into

const client = new DynamoDBClient(
  process.env.ENVIRONMENT === "local"
    ? {
        endpoint: "http://dynamodb-local:8000",
      }
    : {}
);
Enter fullscreen mode Exit fullscreen mode

Run and Test Locally

Finally, we can start running and testing our project locally using AWS SAM.

AWS SAM provides several commands for us to run our function locally like:

  • sam local invoke

This command is used to initiate a one-time invocation of an AWS Lambda function locally.

$ sam local invoke
Invoking app.lambda_handler (python3.9)
Local image is out of date and will be updated to the latest runtime. To skip this, pass in the parameter --skip-pull-image
Building image....................................................................................................................
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

Mounting /Users/.../sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df Version: $LATEST
END RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df
REPORT RequestId: 64bf7e54-5509-4762-a97c-3d740498d3df  Init Duration: 1.09 ms  Duration: 608.42 ms       Billed Duration: 609 ms Memory Size: 128 MB     Max Memory Used: 128 MB
{"statusCode": 200, "body": "{\"message\": \"hello world\"}"}%
Enter fullscreen mode Exit fullscreen mode
  • sam local start-api

This command is used to run your AWS Lambda functions locally and test through a local HTTP server host. This type of test is helpful for Lambda functions that are invoked by an Amazon API Gateway endpoint.

$ sam local start-api

Initializing the lambda functions containers.
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

 * Running on http://127.0.0.1:3000
Enter fullscreen mode Exit fullscreen mode
  • sam local start-lambda

This command is used to invoke your AWS Lambda function through the AWS Command Line Interface (AWS CLI) or SDKs. This command starts a local endpoint that emulates AWS Lambda.

$ sam local start-lambda
Initializing the lambda functions containers.
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.9-rapid-x86_64.

 * Running on http://127.0.0.1:3001
2023-04-13 07:25:43 Press CTRL+C to quit
Enter fullscreen mode Exit fullscreen mode

Because we want to emulate Amazon API Gateway and AWS Lambda services, we will use sam local start-api and using our dynamodb container network dynamodb_default

sam local start-api --docker-network dynamodb_default
Enter fullscreen mode Exit fullscreen mode 
Initializing the lambda functions containers.
Local image is up-to-date
Local image is up-to-date
Local image is up-to-date
Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

Containers Initialization is done.
Mounting getAllItemsFunction at http://127.0.0.1:3000/ [GET]
Mounting putItemFunction at http://127.0.0.1:3000/ [POST]
Mounting getByIdFunction at http://127.0.0.1:3000/{id} [GET]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected   
instantly/automatically. If you used sam build before running local commands, you will need to re-run sam build for the changes to be picked up. You only need to restart  
SAM CLI if you update your AWS SAM template
2023-11-17 00:32:55 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:3000
2023-11-17 00:32:55 Press CTRL+C to quit
Enter fullscreen mode Exit fullscreen mode

sam local start-api will emulate our function through this endpoint

Mounting getAllItemsFunction at http://127.0.0.1:3000/ [GET]
Mounting putItemFunction at http://127.0.0.1:3000/ [POST]
Mounting getByIdFunction at http://127.0.0.1:3000/{id} [GET]
Enter fullscreen mode Exit fullscreen mode

So we can test POST item by running this command in new terminal.

$ curl -X POST -H "Content-Type: application/json" -d '{"id":"3","name":"test"}' http://localhost:3000
{"id":"3","name":"test"}
Enter fullscreen mode Exit fullscreen mode

And, test GET all items using this command

$ curl http://localhost:3000
[{"name":"test","id":"1"},{"name":"test","id":"2"}]
Enter fullscreen mode Exit fullscreen mode

Run and Test Locally with Localstack

With Localstack the step to run and test locally will be more easier.

But first we need to configure our template.yaml again.

Globals:
  Function:
    Environment:
      Variables:
        ENVIRONMENT: "localstack"
Enter fullscreen mode Exit fullscreen mode

we are changing our ENVIRONMENT variable into localstack so that our DynamoDBCLient will connect directly inside Localstack DynamoDB.

Then we will modify the output of our endpoint into this so that it will output Localstack endpoint that we can use to test our application.

Outputs:
  WebEndpoint:
    Description: "API Gateway endpoint URL for Prod stage"  
    Value: !Sub "http://${ServerlessRestApi}.execute-api.localhost.localstack.cloud:4566/Prod/"
Enter fullscreen mode Exit fullscreen mode

Install samlocal

To emulate AWS services locally with Localstack, we can install and use samlocal, a wrapper for the AWS SAM CLI, so that every aws sam commands will be executed against the Localstack endpoints (http://localhost:4566 by default) instead of real AWS endpoints.

You can install it using these commands.

pip install aws-sam-cli-local
Enter fullscreen mode Exit fullscreen mode

Deploy project to Localstack

To deploy our project to Localstack, we can run this command

samlocal deploy
Enter fullscreen mode Exit fullscreen mode

samlocal will deploy your project but not into AWS but into Localstack.

                Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-b1dc8c32
                A different default S3 bucket can be set in samconfig.toml
                Or by specifying --s3-bucket explicitly.
        Uploading to 9a5628197c0aebb840bb53c26bf4e03c  11801 / 11801  (100.00%)
File with same data already exists at 9a5628197c0aebb840bb53c26bf4e03c, skipping upload
File with same data already exists at 9a5628197c0aebb840bb53c26bf4e03c, skipping upload

        Deploying with following values
        ===============================
        Stack name                   : serverless-api
        Region                       : ap-southeast-1
        Confirm changeset            : True
        Disable rollback             : False
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-b1dc8c32
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================

        Uploading to 1245f569e29a8187dfbd04b64f50e277.template  3166 / 3166  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     
Operation                                         LogicalResourceId                                 ResourceType                                      Replacement
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     
* Modify                                          putItemFunctionApiPermissionProd                  AWS::Lambda::Permission                           False
* Modify                                          putItemFunction                                   AWS::Lambda::Function                             False
* Modify                                          getAllItemsFunction                               AWS::Lambda::Function                             False
* Modify                                          getByIdFunctionRole                               AWS::IAM::Role                                    False
* Modify                                          getAllItemsFunctionApiPermissionProd              AWS::Lambda::Permission                           False
* Modify                                          getAllItemsFunctionRole                           AWS::IAM::Role                                    False
* Modify                                          putItemFunctionRole                               AWS::IAM::Role                                    False
* Modify                                          getByIdFunction                                   AWS::Lambda::Function                             False
* Modify                                          ServerlessRestApiProdStage                        AWS::ApiGateway::Stage                            False
* Modify                                          ServerlessRestApiDeploymenta4d359a69a             AWS::ApiGateway::Deployment                       False
* Modify                                          ServerlessRestApi                                 AWS::ApiGateway::RestApi                          False
* Modify                                          getByIdFunctionApiPermissionProd                  AWS::Lambda::Permission                           False
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------     


Changeset created successfully. arn:aws:cloudformation:ap-southeast-1:000000000000:changeSet/samcli-deploy1700159060/1d7348f1
Enter fullscreen mode Exit fullscreen mode

Once your project is deployed into Localstack locally, samlocal will return this output

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Outputs
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Key                 WebEndpoint
Description         API Gateway endpoint URL for Prod stage
Value               http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
Enter fullscreen mode Exit fullscreen mode

Finally, we can test our service through Localstack endpoint http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/ by running this command
So we can test POST item by running this command in new terminal.

$ curl -X POST -H "Content-Type: application/json" -d '{"id":"3","name":"test"}' http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
{"id":"3","name":"test"}
Enter fullscreen mode Exit fullscreen mode

And, test GET all items using this command

$ curl http://a4uowlkucn.execute-api.localhost.localstack.cloud:4566/Prod/
[{"name":"test","id":"1"},{"name":"test","id":"2"}]
Enter fullscreen mode Exit fullscreen mode

Conclusions

While AWS SAM provides a means to run and test serverless services locally, the process is significantly streamlined with Localstack. Localstack not only facilitates the local execution and testing of serverless applications but also extends its capabilities to emulate various other services, thereby enriching our local development environment.

localstack-services

Check out my previous post

AWS Community Builders

Code Anywhere: Configure Your Personal Web-Based Code Editor with AWS CLI or AWS CDK

Jimmy for AWS Community Builders ・ Oct 31

#aws #cli #cdk #lightsail

Top comments (0)

Read next

notarena profile image

Most Commonly Asked DSA Interview Questions

Notarena -

dev_vaayen profile image

[DevLog #01] Gmail-TUI: Replicating The Gmail-Web Experience In Terminal

Vaayen -

krinskumar profile image

OSD - lab - 6 - Code Formatter

Krinskumar Vaghasia -

ujjwall-r profile image

How Real-Time Editing Works: Understanding Event Ordering in Distributed Systems

Ujjwal Raj -