Challenge #3: Using Offline Tools to speed up dev in Serverless

#challenge #aws #python

Quick Recap

In the first two challenges, we created a basic API-based loyalty application and we upgraded our app to handle batch creating loyalty cards by uploading files to S3. Each restaurant under our company can now upload a CSV of old loyalty cards and have it batch-created in the system instead of filling up the form one by one.

Your Next Challenge

We have shifted 90% of our old paper-based loyalty cardholders to our new upgraded system with your CSV upload module. Your manager is happy, and so are your customers. But deep down, you know something is wrong. You are not satisfied with your work.

You write code, but there's no way to test locally. You have to use serverless deploy every time. Need to correct that syntax error? Deploy first and invoke manually. It takes you 3-5 minutes to get feedback for such a small change. It's driving you nuts!

In this challenge, we will introduce three ways to test locally.

Specification

(1) Use Serverless Offline

In the past two challenges, we created an API webserver, but we haven't been able to test our APIs locally. We deployed straight away. In frameworks like Flask and Ruby on Rails, we usually get to work with a local web server before deploying.

Using serverless offline, upgrade your serverless.yml so that you can run a local version of your API and test your API via localhost:8080.

You can learn more about serverless offline here.

(2) serverless invoke local

This command lets you invoke your Lambda function locally through an emulator that comes prepackaged with Serverless Framework. You simply run the command below every time you want to trigger your function.

serverless invoke local --function functionName

This command is pretty straightforward to use when testing out lambda functions that don't need you to send any input (i.e the Get All cards endpoints). But what if you need to send the card information to create a card?

First, we create a file inside a mock file inside the mocks folder (let's name it mocks/create_card.json):

{
    "body": "{\"CardName\": \"Raphael Jambalos\", \"CardNumber\": \"4386290100000200\"}"
}

Now, let's refer to it by adding the --path parameter to our earlier command:

serverless invoke local --function create_card \
                        --path mocks/create_card.json \
                        --region ap-southeast-1

You should now be able to test your functions.

I think one question is that when do you use serverless invoke over serverless offline. I usually use the invoke command when the application I'm developing is not API-based - when lambda functions are triggered by other event sources like scheduled triggers or S3 file uploads.

You can learn more about it here: https://www.serverless.com/framework/docs/providers/aws/cli-reference/invoke-local/

(3) Using DynamoDB Offline

Even if we use serverless offline and serverless invoke local to run our functions locally, we are still referring to AWS resources in the cloud (such as the DynamoDB table that stores the loyalty cards we created). While it might not be a huge inconvenience to continue using DynamoDB in the cloud, it comes with a few caveats:

we have to deploy every time our schema changes in DynamoDB
we incur charges in our AWS account (for most cases, this should be below 5USD. Except if you're doing stress testing - loading and reading thousands of rows of data often)
if there's no internet, we cannot connect to the DynamoDB table (and hence, development stops)

For this subchallenge, upgrade your serverless.yml file to use DynamoDb offline when you are developing. Install and run the DynamoDB client in your local machine. It should give you a local endpoint (i.e localhost:5000) and you should connect you serverless application there (by manipulating your serverless.yml)

You can learn more about it here: https://www.serverless.com/plugins/serverless-dynamodb-local

What we have done

We used tools that allowed us to develop our serverless application with local tools. This speeds up development because we no longer have to communicate with AWS services in the cloud as we develop.