Create and deploy a To-do CRUD service using Node.js, AWS and Serverless Framework

Introduction

In this post, we will go through creating a simple CRUD (Create, Read, Update and Delete) service by creating a To-do service using Node.js, AWS and Serverless Framework. We will be creating and deploying AWS Lambda functions and expose them as RESTful APIs using Amazon API Gateway. We will also make use of a powerful Node.js middleware middy to make our development even simpler.

Architecture

Below is a high-level architecture of what we are going to build.

Setup

Before we get started, we will require some setup.

Create AWS Account

We must create an AWS account. For this set of instructions, it will not cost us anything. The AWS Free Tier should be plenty for our use case.

Serverless Framework Installation

We will install the Serverless Framework on our machines as a standalone binary.
There are multiple ways of doing this in the Serverless docs. In this post, we will be installing through NPM:

npm install -g serverless

To verify installation we will execute:

sls --version

AWS CLI Installation

In order to use the Serverless Framework efficiently in our machine, we will make use of the AWS CLI. Instructions specific to your machine can be found here. For macOS users like me, the instructions will be:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"

sudo installer -pkg ./AWSCLIV2.pkg -target /

We can then verify the installation as follows:

aws --version

Configuring AWS CLI

At the moment AWS CLI does not know who we are. We will need to provide some information about this. The purpose of this is to link our local machine CLI with AWS.

Going back to our AWS console. We go into the Identity and Access Management (IAM) service. This service manages who can access our AWS resources.

Click on the "Users" tab.

Then, we can create a user.

Select "Attach existing policies directly". For the purpose of this post, we will grant this user with AdministratorAccess. Ideally, we should only grant users the level of access that is required.

The step to add tags can be skipped for this tutorial and we can proceed with creating the user.

Take note of your AWS Management Console access sign-in link. Note that the prefix on the link is our created user ID.

Also, take note of your Access Key ID and Secret Access Key.

Back in our terminal, we will execute the following command then enter the credentials we created. We will then select the location appropriate for us. In my case, I chose Europe as it is closest to me and that is where I would like my data to be stored.

aws configure

Now, AWS is configured and linked to our local machine.

Create Project

Now, we will create a project, which we will call todo-service. We will use a fork of a base project from Ariel Weinberger at codingly.io.

sls create --name todo-service --template-url https://github.com/jericopingul/sls-base

This will create a starter project for us. We have called it todo-service because all operations we will be doing for this to-do API will be defined within this service.

In our serverless.yml, we will add our region within the provider property. In my case it will be:

provider:
  stage: ${opt:stage, 'dev'}
  region: eu-west-1

You may be curious what the stage property is. In this case, this will define the stage to which where we will deploy our service. In real-life there will be multiple stages that include the likes of production or any other stage, depending on the development requirements. In this tutorial, we will just use one stage dev.
In terms of the syntax, the opt.stage can be used to reference variable, while the second parameter is a default ('dev') if opt.stage is not set.

We also use two plugins:

plugins:
  - serverless-bundle
  - serverless-pseudo-parameters

serverless-bundle provides us with a number of benefits including allowing us to bundle our JavaScript using webpack, reduce our bundle size, allow the use of modern JavaScript (ES6+) with minimal configuration.

serverless-pseudo-parameters allows us to easily interpolate AWS parameters which will make our life easier later on. More information on this plugin can be found here.

Create a Database

We will need to store our to-do items in a database. We will make use of a NoSQL DynamoDB provided by AWS. The AWS free tier gives us a generous amount of storage.

In order to create the database, we will add the following statement to our serverless.yml so that we can instruct CloudFormation to create it in AWS. We define an attribute that is going to be our primary key, in this case, it is id.

provider:
  ...

resources:
  Resources:
    TodoTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: TodoTable-${self:provider.stage}
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH

In the above statement, we are instructing AWS CloudFormation to create a table named TodoTable-dev with a primary key id with a PAY_PER_REQUEST billing.

With the changes above, we can create the database on AWS and deploy our project by using the command:

sls deploy -v

We are using an optional -v option which means verbose just to see more information on the logs.

In AWS CloudFormation we should see the todo-service-dev stack. In the resource tab, we can verify that our table has been created:

Third-party libraries

AWS SDK

We will require the aws-sdk library to create the DynamoDB client instance. More information here.

Middy

We will be using the middy middleware library to simplify our AWS Lambda code. We will be using middy, middy body-parser and middy HTTP error handler. So we will install the following:

yarn add @middy/core @middy/http-event-normalizer @middy/http-error-handler @middy/http-json-body-parser

The purpose of each library are as follows:

• @middy/core is the core middy library.
• @middy/http-event-normalizer simplifies accessing query string or path parameters.
• @middy/http-error-handler handles uncaught errors and generates a proper HTTP response for them. See more info here.
• @middy/http-json-body-parser parses HTTP requests with a JSON body and converts them into an object for use within our Lambdas.

Error handler

@middy/http-error-handler recommends using http-errors library to be used together with their library to simplify creating errors so we will also install the following:

yarn add http-errors

UUID

We will require to generate a unique identifier for each of our to-dos in the database so we will use the uuid library.

yarn add uuid

Creating our AWS Lambdas

Now, we will move onto creating our AWS Lambdas that we will expose via HTTP.

Create a To-do

Now we will create our create to-do Lambda function. In our serverless.yml we will add the following entry in the functions property:

functions:
  createTodo:
    handler: src/handlers/createTodo.handler
    events:
      - http:
          method: POST
          path: /todo

This means that we will have a createTodo.js file that exports a function handler in the src/handlers directory.

Here, we will use the middleware we installed. We will define a common middleware for all of Lambdas we will use in our project in common/middlware.js with the contents:

import middy from '@middy/core';
import jsonBodyParser from '@middy/http-json-body-parser';
import httpEventNormalizer from '@middy/http-event-normalizer';
import httpErrorHandler from '@middy/http-error-handler';

export default (handler) =>
  middy(handler).use([
    jsonBodyParser(),
    httpEventNormalizer(),
    httpErrorHandler(),
  ]);

This exported function will execute the listed middlewares in the array on the passed handler function.

Below, we add the custom property in our serverless.yml file. The purpose of this is to make it easier to make change our tables down the line. We make use of AWS CloudFormation intrinsic functions Ref and GetAtt so that when our stack is deployed then these values will be dynamically evaluated. For the purpose of this post, we will turn off linting on our JavaScript code but I would recommend this to be turned on in production code.

custom:
  TodoTable:
    name: !Ref TodoTable
    arn: !GetAtt TodoTable.Arn
  bundle:
    linting: false

We will also require to add permissions to our Lambda in serverless.yml to create entries in our database table:

provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:PutItem

Below will be the code for our Lambda function in our createTodo.js file. We create a to-do item with the description from the request body and we set the done status as false by default. Note that we execute our common middleware in the last line.

const dynamoDB = new AWS.DynamoDB.DocumentClient();

async function createTodo(event, context) {
  const { description } = event.body;
  const now = new Date();

  const todo = {
    id: uuid(),
    description,
    created: now.toISOString(),
    updated: now.toISOString(),
    done: false,
  };

  try {
    await dynamoDB
      .put({
        TableName: process.env.TODO_TABLE_NAME,
        Item: todo,
      })
      .promise(); // to return a promise instead
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 201,
    body: JSON.stringify(todo),
  };
}

export const handler = middleware(createTodo);

We can deploy our changes with the same deploy command:

sls deploy -v

We should find our API URL/endpoint that we created in our terminal and we can verify using a REST client, here I'm using postman:

Retrieve To-dos

We create a new entry in serverless.yml to add the new getTodos function:

functions:
  ...     
  getTodos:
    handler: src/handlers/getTodos.handler
    events:
      - http:
          method: GET
          path: /todo

We are also required to add Scan action permissions.

provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Scan

Below is the code to retrieve all entries in the database table then returns it.

async function getTodos(event, context) {
  let todos;

  try {
    const result = await dynamoDB
      .scan({
        TableName: process.env.TODO_TABLE_NAME,
      })
      .promise();
    todos = result.Items;
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
    body: JSON.stringify(todos),
  };
}

export const handler = middleware(getTodos);

Update a To-do

We will require to add the UpdateItem permissions.

provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:UpdateItem

We create the following new function in our functions property. Note that we are using PATCH as we are going apply a partial update to the resource.

functions:
  ...
  updateTodo:
    handler: src/handlers/updateTodo.handler
    events:
      - http:
          method: PATCH
          path: /todo/{id}

Below we have the code for our update function. We will only allow the description and done fields to be updated. In the implementation below, we require at least one of description and done to be part of the request body, updates the data accordingly and finally returns the updated resource.

async function updateTodo(event, context) {
  const { id } = event.pathParameters;
  const { description, done } = event.body;
  const now = new Date();

  if (!description && done === undefined) {
    throw new createError.BadRequest(
      'You must update either description or done status!'
    );
  }

  const updatedAttributes = [];
  const expressionAttributeValues = {};

  if (description) {
    updatedAttributes.push(`description = :description`);
    expressionAttributeValues[':description'] = description;
  }

  if (done !== undefined) {
    updatedAttributes.push(`done = :done`);
    expressionAttributeValues[':done'] = !!done;
  }

  updatedAttributes.push(`updated = :updated`);
  expressionAttributeValues[':updated'] = new Date().toISOString();

  const updateExpression = `set ${updatedAttributes.join(', ')}`;

  const params = {
    TableName: process.env.TODO_TABLE_NAME,
    Key: { id },
    UpdateExpression: updateExpression,
    ExpressionAttributeValues: expressionAttributeValues,
    ReturnValues: 'ALL_NEW',
  };

  let updatedTodo;

  try {
    const result = await dynamoDB.update(params).promise();
    updatedTodo = result.Attributes;
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
    body: JSON.stringify(updatedTodo),
  };
}

Delete a To-do

We first add the DeleteItem permission:

provider:
  ...
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:DeleteItem

Then add the new function in our functions property in serverless.yml:

functions:
  ...
  deleteTodo:
    handler: src/handlers/deleteTodo.handler
    events:
      - http:
          method: DELETE
          path: /todo/{id}

Below we have our delete function that simply deletes an entry in the database table based on the id.

async function deleteTodo(event, context) {
  const { id } = event.pathParameters;

  const params = {
    TableName: process.env.TODO_TABLE_NAME,
    Key: { id },
  };

  try {
    await dynamoDB.delete(params).promise();
  } catch (error) {
    console.error(error);
    throw new createError.InternalServerError(error);
  }

  return {
    statusCode: 200,
  };
}

Closing Notes

We have created a simple to-do CRUD service using Node.js, AWS Lambda and Serverless Framework. We have also made use of middleware libraries to simplify the development of our Lambdas.

There are a number of steps involved in the initial set up but once we have done this, it is straightforward to add create and add new functions.

Thank you for following along and I hope that this simple CRUD service helps in creating any serverless project. ✌🏼