Learn about packaging your NodeJS-based AWS Lambda functions with esbuild for improved cold start times and reduced deployment package size
This article was originally posted on my development blog, where you can find other articles about TypeScript, AWS, Lambda and open source.
Introduction
AWS SAM is a great way to package and deploy your AWS Lambda functions, but as your project grows, it's easy to become frustrated with increasing deployment times from an ever-growing deployment package containing all your functions and NodeJS dependencies.
The serverless framework supports an excellent workflow with the serverless-webpack plugin to package your functions individually with webpack. For AWS SAM, it's probably worth checking out the aws-sam-webpack-plugin, but I've found it far simpler to configure esbuild to perform the same task.
Why package my code using a bundler?
- Improved cold start time - smaller deployment packages generally means less time copying your deployment package to the AWS Lambda container each time it has to be started
- TypeScript support - NodeJS doesn't support TypeScript natively, so converting it to JavaScript can be done as part of a bundilng step
- Tree-shaking and reduced deployment package size - a bundler will only include the node dependencies your function imports, and if enabled correctly for tree-shaking, your overall bundle size will be reduced to just the blocks of code that are needed (which is great for multi-functional libraries like lodash which are rarely used as a whole).
Disadvantages
Using a bundler does come with some downsides, including:
-
Increased complexity - bundling your code means more steps before running your code locally using
aws sam local
and more deployment time steps - Inceased packaging time - as your functions are packaged individually, a unique deployment package is generated for each one, instead of uploading a single deployment package with all of your functions. However, this may still be acceptable as each function's deployment package will be considerably smaller after bundling
-
Poor stack trace support - getting decent stack trace support with bundled code can be very difficult, especially for exceptions. In many cases, you'll still be debugging by
console.log()
or guessing based on the function names in your stack trace. -
Module support - some node modules are not compatible with bundling, either because they assume the existence of the
package.json
andnode_modules
folder (looking for filesystem artefacts in their code), or they are trying to perform auto-instrumentation for telemetry (which bundling interferes with).
esbuild vs webpack
I've used webpack before for bundling, and while it is a mature and flexible bundler, you may run into problems with its memory usage and execution when you use it on very large projects with dozens of AWS Lambda functions. This is because it is written in JavaScript and must execute the same bundling process for each Lambda function.
esbuild is a newer bundler built in Go, which is gaining considerable popularity for its speedy execution time and simple configuration, as well as removing the need for Babel with its native TypeScript support. However, it should be noted it has less features and is less mature than webpack (and probably always will because it is deliberately targeting a reduced feature set).
I've chosen esbuild because I've found it mature enough to use in production for large, established AWS Lambda based applications, and it reduced bundling times from over 5 minutes to less than 30 seconds.
Using esbuild with AWS SAM
Configuring your project
In this setup, we will bypass sam build
and set up an esbuild bundling step that must be run before every sam package
/sam deploy
.
Install esbuild
First get esbuild installed with npm
or yarn
:
npm i esbuild -D
# OR
yarn add esbuild -D
Source code layout
Next structure your function entry points to be hosted in individual directories - this helps with ensuring that bundled output for each function gets its own directory, and hence can be deployed individually by SAM.
e.g.
src/say_hello/index.ts
src/send_email/index.ts
(I've assumed TypeScript here, but you can use JavaScript by simply changing the file extension. Remember that you should have typescript
installed in your project and a tsconfig.json
set up for your Node version if you're transpiling TS).
If you have any other shared code, it should be put into another directory e.g /shared
or /util
and simply imported.
Setup esbuild configuration
Although esbuild can be run from the command line, a quick JavaScript configuration file is easier to maintain and read with the number of options we will use.
const fs = require('fs');
const path = require('path');
const esbuild = require('esbuild');
const functionsDir = `src`;
const outDir = `dist`;
const entryPoints = fs
.readdirSync(path.join(__dirname, functionsDir))
.map(entry => `${functionsDir}/${entry}/index.ts`);
esbuild
.build({
entryPoints,
bundle: true,
outdir: path.join(__dirname, outDir),
outbase: functionsDir,
platform: 'node',
sourcemap: 'inline',
});
This assumes your Lambda function entry points are located in ./src/<function_name>/index.ts
and will be built to ./dist/<function_name>/index.js
.
If you would prefer to do the above using the command line, the following esbuild
line should be equivalent:
npx esbuild \
--bundle src/*/index.ts \
--outdir=dist \
--outbase=src/ \
--sourcemap=inline \
--platform=node
Setting up your AWS SAM template
Each function in your AWS SAM template will need to be updated to use the new package layout:
Resources:
HelloFunction:
CodeUri: dist/say_hello
# assumes your function is at `src/say_hello/index.ts` or
# `src/say_hello/index.js` with an exported entry point
# function called `handler`
Handler: index.handler
...
SendFunction:
CodeUri: dist/send_email
Handler: index.handler
...
Building and deploying your function
You can now build and deploy your function by running esbuild before each sam package and sam deploy step i.e.
node esbuild.js
sam package
sam deploy
Running locally
Your bundled code can be executed locally, but you need to run node esbuild.js
before you call sam local invoke
.
If you want to re-run esbuild in watch mode, you can modify your esbuild.js
accordingly:
esbuild.
build({
...
watch: process.argv.includes('--watch'),
});
and run node esbuild.js --watch
in the background (or add --watch
to the command line variant given in Setup esbuild configuration).
Top comments (0)