Integrating SAM Into Container Workflows With http-lambda-invoker

In this article I'm going to look at how we can execute Lambda functions via API in an existing docker-compose workflow for use in local development or continuous integration.

tl;dr

Check out my solution on docker hub or github, and the example repo.

Table of Contents

• SAM and docker-compose
• Community Support
• ExpressJS Solution
• Golang Solution
• Putting It All Together
• Next Steps

SAM and docker-compose

It's no secret that I love the SAM framework and I've been making good use of it in my professional life. One sticking point for me has been that the teams I work have a lot of existing workflows that resolve around docker-compose.

version '3.7'

services:
  our-service:
    image: our-registry/our-service:latest
    environment:
      - DB_HOST=db
    ports:
      - "8080:8080"
  db:
    image: our-registry/our-seeded-db:latest
    ports:
      - '5432:5432'
  // etc

So how can SAM work with a docker-compose based workflow like this one? Well, we can just run sam local start-api. Now instead of kicking off everything with a single command, we're running two commands. That's not the end of the world, but the elegance of a single command is actually pretty powerful when you need large and varied development teams to work this way and be productive on the same stack.

A larger issue is just the way SAM works. On each API request, SAM will start a Docker container, execute the request, then shut down and remove the container. This is great for a one-off execution of the Lambda function, but it will not make for a great development experience when compared to persistent containers. It's also just not how Lambda works. Lambda functions will stick around for some time after being invoked and the same Lambda function can run again.

Community Support

I'm not the only one to have noticed this. There's been an open issue on github for some time. That thread offers the solution of running the underlying Docker container in stay-open mode. This works extremely well for keeping the function callable and loaded, but unfortunately it doesn't interop with the built-in REST gateway that the SAM framework uses.

When you start SAM with sam local start-api, the framework will spin up a flask app on your workstation that triggers the docker-lambda image. Again, this works great for a single invocation, but the flask app doesn't understand the DOCKER_LAMBDA_STAY_OPEN flag that can be given to docker-lambda. Follow all of that? If not, maybe this issue I opened will shed some light on the situation. In that thread, you'll see the author of docker-lambda, Michael Hart, pointing me to serverless offline.

serverless offline still isn't quite what I'm looking for. Remember, I want to add to an existing docker-compose, not build in a new framework. Maybe I missed something there, but going through the examples, I didn't see what was I looking for. Feel free to set me straight in the comments.

ExpressJS Solution

Since I couldn't find exactly what I was looking for and don't believe in demanding open-source maintainers to provide the features that I need but they might not, I decided to write my own proxy gateway and publish it to Docker hub. In order to get up and running quickly, I wrote a quick api that delegates to Lambda invoke using express (and TypeScript of course).

My express version of this app worked fine, but I got exposed to the gnarly underbelly of express and how cookies are handled. I wound up writing this code:

/*
 * Lambda invocation response mapped to express response object.
 */
const lambda2ExpressResponse = (response: Lambda.InvocationResponse, res: Response): void => {
  const payload = JSON.parse(response.Payload?.toString() || '');
  // Loop over headers and set them in express response object
  Object.entries(payload.headers).forEach(([k, v]) => {
    // To support multiple cookies, parse them individually and remap to res.cookies one-by-one.
    if (k.toLowerCase() === 'set-cookie') {
      // cookie parts are separated by ;
      const cookieParts = (v as string).split(';');
      // the first part will be the name and value separated by =
      const [cookieName, cookieVal] = cookieParts[0].split('=');
      // remaining parts are the cookie options separated by =
      const cookieOptions = cookieParts?.slice(1)?.reduce((prev, curr) => {
        const part = curr.split('=');
        // AWS makes everything PascalCase so turn it back to camelCase to use with express.
        const optionName = camelCase(part[0].trim());
        // Some options may not have a value, assume the value should be true.
        const optionVal = part[1]?.trim() || true;
        // cookie option types are string or coercible booleans except `expires` which must be a Date.
        if (optionName === 'expires') {
          return { ...prev, [optionName]: new Date(optionVal as string) };
        }
        return { ...prev, [optionName]: optionVal };
      }, {});
      res.cookie(cookieName, cookieVal, cookieOptions);
    } else {
      res.set(k, v as string);
    }
  });
  res.status(payload.statusCode).send(payload.body);
};

Golang Solution

Eew, that's terrible. There has to be a better way to translate headers. Anyway, I'd been wanting to learn golang and this seemed like a good time to try. I'm not saying my code is perfect, but it was reasonably easy to write, works just as well and my original 137MB image is now below 20MB! Also headers are handled like this:

    // Add headers to ResponseWriter omitting content-length, which came back with the wrong length.
    for key, value := range response.Headers {
        if key != "content-length" {
            w.Header().Add(key, value)
        }
    }
    // Enable cors
    w.Header().Set("Access-Control-Allow-Origin", "*")

So now to move onto an example. I assume that if you've read this far, you understand the problem I'm trying to solve. Consider I have a legacy application that I'm supporting and I want to extend it by adding a user-facing Lambda function. Here's a simple docker-compose file to represent that application:

version: '3.7'

services:
  legacy:
    container_name: legacy
    build: legacy
    ports:
      - '8000:8000'
  static:
    container_name: static
    image: nginx
    ports:
      - '80:80'
    volumes:
      - ./public:/usr/share/nginx/html

This will launch my web page hosted in an nginx container running on port 80. API calls will be mapped to localhost on port 8000 to talk to my legacy service. Now I want to add a Lambda function running on port 8080. I can use docker-lambda to run a persistent version of my Lambda.

  lambda:
    container_name: lambda
    image: lambci/lambda:nodejs12.x
    command: app.HttpLambdaInvokerExample
    environment:
      - DOCKER_LAMBDA_STAY_OPEN=1
      - TZ=America/New_York
    volumes:
      - ./build:/var/task:ro,delegated

This is assuming my Lambda source is in ./build (like I suggested in another post). If we wanted to use a plain bootstrapped SAM project, we'd map volumes to ./.aws-sam/build:/var/task:ro,delegated.

Adding that to my docker-compose will run the lambda function in stay-open mode, but there's no way to call via http. To do that, I'll need to add a container for http-lambda-invoker.

  api:
    container_name: api
    image: elthrasher/http-lambda-invoker
    environment:
      - LAMBDA_ENDPOINT=http://lambda:9001
      - LAMBDA_NAME=HttpLambdaInvokerExample
      - PORT=8080
    ports:
      - '8080:8080'

This container will receive http traffic on port 8080 and invoke the Lambda function with an ApiGatewayProxyEvent, just as though it had been invoked via API Gateway.

Putting It All Together

I wrote a really simple and dumb frontend to prove I can call into my legacy and my serverless backends.

  <body>
    <h1>http-lambda-invoker demo</h1>
    <div>Data from lambda: <span id="lambda"></span></div>
    <div>Data from legacy: <span id="legacy"></span></div>
  </body>

Now I can run docker-compose up -d to start all my containers and see my front end connecting successfully to both my legacy backend and my new serverless backend.

Check out the example repo.

Next Steps

If you're like me and very comfortable with Docker and docker-compose, then you may prefer a solution like this that gives you more control over the inner workings of the framework. But for many others, the simplicity of SAM is all you'll ever need. I recommend always validating your use case before you write another big yaml document.

This solution is very much an MVP. To make this more robust, I would consider adding these features:

• Parse SAM templates to support routing to functions.
• Support both REST and HTTP Gateway.
• Support other kinds of events.

Cover: Alexander Dmitrievich Litovchenko( 1835 - 1890) "Charon carries souls across the river Styx"