Challenge #4: Create CI/CD for Serverless Apps

This was presented as a lightning talk last Nov 17, 2022 in the 4th AWSUG F2F Meetup at the GCash Office, BGC

In this post, we will create a basic CI/CD pipeline for the serverless application we built in the first challenge. For a quick recap, our serverless application has 3 API endpoints: for creating a loyalty card, seeing all loyalty cards, and displaying details about one loyalty card. The loyalty card data are stored in DynamoDB:

With a CI/CD pipeline, we can deploy changes to our application in a consistent and automated way.

To achieve that, we will create a CI/CD pipeline with AWS CodePipeline. Inside CodePipeline, we have 2 stages. One for Source and another for Build. By the end of this walkthrough, our changes will be deployed to production once it is merged to master:

• SOURCE STAGE: CodePipeline simply watches for any changes from the master branch in GitHub. Once a change has been pushed, our pipeline gets triggered and the code is copied for further processing
• BUILD STAGE: CodePipeline triggers CodeBuild and supplies it with a copy of the source code. CodeBuild is an automated build service. It relies on a file called "buildspec.yml" inside the repository. The file contains instructions on how to prepare and deploy our Serverless App. When CodeBuild is triggered, it executes these instructions.

Now, let's set up our CI/CD Pipeline

[1] Head over to CodePipeline by searching for it in the top search bar

Then, hit the "Create pipeline" button

[2] Type the name of the pipeline and leave everything else as default. Press Next

[3] In the Source stage, select the source provider that applies to you. For me, my code is stored in GitHub so I select the "Github (Version 2)" option.

Since my repository is outside AWS, I need to authorize CodePipeline to access it. Press "Connect to GitHub". You may have to do something similar if you select BitBucket. Aside from the two, CodePipeline supports CodeCommit as a source repository. However, Gitlab is not natively supported.

A pop-up screen will appear. Create a name for your connection. Then, press "Connect to GitHub"

If you are signed in to the GitHub account, you will see this pop-up screen in Github that requires your permission. Grant it by clicking "Authorize AWS Connector for GitHub"

[4] You will be redirected back to this screen. If you have done this before, you will see a pre-installed GitHub App. If there is none, hit "Install a new app". You will also have to click "Install a new app" if your GitHub repository is owned by another organization and your GitHub user was just invited to that organization.

After you have chosen the correct GitHub App, click "Connect"

[5] With that, the pop-up window closes and you are redirected back to the "Add source stage" screen. Choose your repository and branch name. Leave the rest as default and then, click "Next"

[6] In the build provider, choose "AWS CodeBuild". If you haven't created your CodeBuild Project yet, hit "Create Project"

Another pop-up screen appears for us to create our CodeBuild project. In this screen, add your project name:

Scroll a bit down for the environment section.

• Environment image: Managed image
• Operation System: Ubuntu
• Runtime: Standard
• Image: aws/codebuild/standard:5.0 (or whatever the latest version is)
• Image version: Always use the latest image for this runtime version
• Environment type: Linux
• Privilged: Enabled

In the previous step, we connected GitHub to our CI/CD pipeline. In this step, we created a CodeBuild project that runs a series of commands to build our application. These commands are listed under the buildspec.yml file. Here is our project's buildspec.yml file

In our project's buildspec.yml, we only used 2 phases: install and build. Phases allow us to group our build commands. For our install phase, we installed serverless framework and pip. Then, for the build phase, we ran serverless deploy to deploy our application.

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.8
    commands:
      - ls -a ~
      - npm install
      - npm install -g serverless
      - curl -O https://bootstrap.pypa.io/get-pip.py
      - ls -a ~
      - python3 get-pip.py --user
      - pip --version
  build:
    commands:
      - serverless deploy --region $AWS_REGION

Leave the rest as default and click "Continue to CodePipeline". This creates the CodeBuild project. You will then be redirected back to the CodePipeline screen, with a pre-filled form. Double-check the values. Once satisfied, click next:

[7] You will be redirected to the deploy stage. We don't need this for now. With the serverless deploy command in CodeBuild, we are deploying the application already. Click "Skip Deploy Stage".

Then, we will be on the Review screen. Review the values and press "Create pipeline". Right after pipeline creation, it executes immediately.

The Source stage executes immediately and fetches code from GitHub. If this stage fails the first time, just click retry. Sometimes the first run malfunctions.

Once the source stage is done, the Build stage triggers. In this stage, CodeBuild executes the instructions coded inside your buildspec.yml. If you are curious as to what is happening while CodeBuild is executing, click the "details" and you will see a log of the deployment:

It will take a few minutes before the build completes. Wait for a few minutes and come back to it.

Let's troubleshoot the deployment

[8] From the result of our first run, it looks like the execution failed.

To figure out what's wrong, scroll down and find the last few lines of the execution. You shall see the error there.

It looks like our Build execution failed when we tried to deploy our serverless app using the serverless deploy command. The error is the role that CodeBuild uses is "not authorized to perform" operations in CloudFormation. Since Serverless Frameworks apps are just CloudFormation deployments under the hood, we need to equip our CodeBuild with an IAM role that has permission to access CloudFormation and other needed resources.

[9] Let's update the permission of our CodeBuild role. First, let's go to the Build Project that powers our execution.

Then, under Edit, click Environment:

You will see this screen. Copy the ARN of the Service role:

Then, go to the AWS IAM console.

Click "Roles" on the left-hand side menu and search for the role

Under Permissions, click Attach policies.

Search for the following AWS-managed policies and attach them to the role. In production, be more deliberate and granular about the permissions we grant to this role. For our exercise, this will do.

• AmazonS3FullAccess
• AWSCloudFormationFullAccess
• AmazonDynamoDBFullAccess
• AmazonAPIGatewayAdministrator
• CloudWatchLogsFullAccess
• IAMFullAccess
• AWSLambda_FullAccess

Now, we're good. Let's run the CodePipeline again. Find the pipeline you created earlier and click "Release Change"

On this run, our application is now deployed!

Let's see it in action on our Postman collection:

Now, let's try to modify the response body just a bit by adding "status": "success". Then, we commit the change to master and our CI/CD should deploy our change:

Our CI/CD pipeline should automatically run. After a few minutes, our change should now reflect in our API.

That's all folks!

Photo by JJ Ying on Unsplash