Create a Rest API with AWS API Gateway, a Lambda function and a RDS Aurora Cluster deployed by CDK

Alright, I admit, the title of this blog post is rather long. But hey, the advantage is that you already know exactly what we're going to cover in this article. Since we believe that AWS CDK is one of the most advanced tools available today for implementing infrastructure as code on AWS, this article contains only CDK code fragments.

Please note, this is not a complete step-by-step guide. We only want to document the code fragments required to create and connect the AWS resources needed for a Rest API that talks to a RDS database. We assume that you are already familiar with CDK and know how to implement the following fragments into your CDK stacks. And, just to mention that, in our real-world application, we divide resources among different stacks (e.g. a network stack, a database stack, etc.). However, for the sake of simplicity, this will be excluded here.

Ready? Let's dive into it.

VPC

Our database has to be connected to a VPC. That's why we first create VPC and add the needed VPC Interface Endpoints to it.

/* nodejs modules */
import * as ec2 from '@aws-cdk/aws-ec2';

......

/* create a vpc */
const vpc = new ec2.Vpc(this, 'vpc', {
  ...
});

/* Secrets Manager Endpoint */
vpc.addInterfaceEndpoint('sm',{
  service: ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER
});

/* RDS Data API Endpoint */
vpc.addInterfaceEndpoint('rds_data',{
  service: ec2.InterfaceVpcEndpointAwsService.RDS_DATA
});

Database

Next, we add the construct to create our database cluster. Please note, the following example creates a cluster that meets our own specific needs. You will probably need to adjust the cluster properties.

/* nodejs modules */
import { SubnetType } from "@aws-cdk/aws-ec2";
import * as rds from '@aws-cdk/aws-rds';


/* RDS - Aurora - database */
const dbCluster  = new rds.ServerlessCluster(this, 'database', {
  engine: rds.DatabaseClusterEngine.auroraPostgres({version: rds.AuroraPostgresEngineVersion.VER_10_12}),
  vpc: vpc,
  vpcSubnets: {
    subnetType: SubnetType.ISOLATED
  },
  defaultDatabaseName: 'mydatabase',
  enableDataApi: true,  /* this is important ! */
  removalPolicy: cdk.RemovalPolicy.RETAIN,
});

If we create a database cluster with this high level construct, the admin account for accessing the database is created automatically and the credentials are stored in the Secrets Manager.

Here's a little side note. For our application we needed to add cross-region secret replication to the automatically generated secret. It took us some efforts to achieve it, but finally we made it. The first line grabs the CloudFormation node of the secret from the clusters child nodes. Once we have that, the addReplicationRegion can easily be called.

const secret = dbCluster.node.children.filter((child) => child instanceof rds.DatabaseSecret)[0] as rds.DatabaseSecret;
secret.addReplicaRegion('us-east-1');

Alright. What's next?

Lambda function

Let us create a Lambda function that is capable of talking to our database. Our Lambda function will be connected to the same VPC as the database, it will get two environment variables (ARN of the database cluster and ARN of the secret) and we extend the timeout by a few seconds.

Now check out the last line in the following fragment. That's why we love CDK. One single line of code and all the IAM permissions for the Lambda function needed to be able to connect to the database are set exactly as we need it, including the permissions to grab the admin credentials from Secrets Manager.

/* nodejs modules */
import * as lambda from '@aws-cdk/aws-lambda';

const lambdaHandler = new lambda.Function(this, 'myLambdaFunction',{
  runtime: lambda.Runtime.NODEJS_14_X,
  code: lambda.Code.fromAsset('./lib/lambda/myLambdaFunction'),
  handler: 'index.handleApiRequest',
  vpc: vpc,
  environment: {
    'dbClusterArn': dbCluster.clusterArn,
    'secretArn': secret.secretArn
  },
  timeout: cdk.Duration.seconds(30)
});

/* grant permissions to access the RDS Data API */
dbCluster.grantDataApiAccess(lambdaHandler);

Now we provide you with some sample content of the Lambda function itself (./lib/lambda/myLambdaFunction/index.ts). Please note that this code is not suitable for production! It is just a demo to show you how to run a simple SQL command on a database and return the result.

import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
import * as AWS from 'aws-sdk';

export const handleApiRequest = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {

  // prepare SQL command
  const sqlParams = {
    secretArn: process.env.secretArn,
    resourceArn: process.env.dbClusterArn,
    sql: 'select * from myDemoTable;',
    database: 'mydatabase',
    includeResultMetadata: true
  } as AWS.RDSDataService.ExecuteStatementRequest;

  const rdsData = new AWS.RDSDataService();
  const result = await rdsData.executeStatement(sqlParams, (err, data) => {
    if (err){
      console.log(err);
    } else {
      console.log(data);
    }
  }).promise();


  return {
    statusCode: 200,
    body: JSON.stringify({
      message: 'command completed successfully',
      result: result,
      event: event
    }),
  };
};

If you run a test on this Lambda function you should already be able to run SQL commands and get the result back.

API Gateway

Now there is only one piece missing, our Rest API. You may won't believe how easy we can create our API with CDK. Check this out.

/* nodejs module */
import * as apigw from '@aws-cdk/aws-apigateway';


/* create an API */
const apiDemo = new apigw.RestApi(this, 'demoApi');

/* add a resource to the API and a method to the resource */
const demo = apiDemo.root.addResource('demo');
demo.addMethod('GET', new apigw.LambdaIntegration(lambdaHandler));

Three lines of code and we have an API with a /demo resource and a GET method and a Lambda function attached to it. And what about the permissions for invoking the Lambda? Everything is set up automatically. Isn't it awesome?

If you want to add one or more request parameters to a method, have a look at the following PUT sample. This was tricky to figure out and hidden very well in the documentation. In addition, we can add a RequestValidator that already checks at the API Gateway whether the request parameter was passed correctly. This has the advantage that we don't have to implement such a check in the Lambda function and it prevents unnecessary Lambda invocations.

Of course, you can also validate the payload of the requests. Check out the great post by Davide de Paolis.

    const stringParamValidator = new apigw.RequestValidator(this, 'stringParamValidator', {
      restApi: apiDemo,
      requestValidatorName: `stringParamValidator`,
      validateRequestParameters: true
    });

    demo.addMethod('PUT', new apigw.LambdaIntegration(lambdaHandler), {
      requestParameters: {
        'method.request.querystring.myparameter': true
      },
      requestValidator: stringParamValidator
    }); 

Again a side note. The aws-apigateway module has another class that could make it even easier to create Lambda backed Rest APIs. You may want to read about it here.

And here is another addition to the API Gateway topic. In our real-world application, we needed a private Rest API. If you are curious about how to create a private Rest API, one that is accessable only within a VPC, here is our API construct with a policy attached.

/* API Gateway endpoint, only for private Rest APIs needed */
const apigwEndpoint = vpc.addInterfaceEndpoint('apiGw', {
  service: ec2.InterfaceVpcEndpointAwsService.APIGATEWAY
});



/* private API Gateway */
const api = new apigw.RestApi(this, 'demoApi', {
  endpointConfiguration: {
    types: [apigw.EndpointType.PRIVATE],
    vpcEndpoints: [apigwEndpoint]
  },
  policy: new iam.PolicyDocument({
    statements: [
      new iam.PolicyStatement({
          effect: iam.Effect.ALLOW,
          principals: [new iam.AnyPrincipal()],
          actions: ["execute-api:Invoke"],
          //the following creates a reference to the RestAPI itself
          resources: [cdk.Fn.join('', ['execute-api:/', '*'])]
      }),
      new iam.PolicyStatement({
        effect: iam.Effect.DENY,
        principals: [new iam.AnyPrincipal()],
        actions: ["execute-api:Invoke"],
        resources: [cdk.Fn.join('', ['execute-api:/', '*'])],
        conditions: {
          "StringNotEquals": {
            "aws:sourceVpc": vpc.vpcId
          }
        }
      })
    ]
  })
});

Summary

Since it took us a couple of hours to figure this all out, we hope that with this article we can help you get your CDK application up and running faster.

As the fragments are copied out of a much more complex application, we cannot guarantee that an error has not crept in. If something is not working properly or you have any additions, we would appreciate it if you let us know.

Best regards and happy coding.

Andy / cloudxs