Overview of AWS Lambda internal extensions

Before we start - this blogpost is strongly inspired by excellent AWS Lambda extensions: The deep dive series by Julian Wood.

What are AWS Lambda extensions?

AWS Lambda extensions allow you augement your Lambda functions, for instance to more easily integrate your Lambda functions with your favorite (or the ones you can afford!) third party tools and services for monitoring, observability, security, and governance.

Lambda supports both internal and external extensions, with a number of differences between the two:

• External extensions run as an independent process in the execution environment and continue to run after the Lambda function invocation completes. They can be written in any language and can be packaged and deployed as Lambda layers. External exentions can be used to perform tasks such as monitoring, observability, security, and governance. Moreover, they can be used with any Lambda runtime.

• Internal extensions run within the runtime, in-process with your code as a separate thread. There are two ways an internal extension is accessed by a function:

  • in-process mechanism (such as NODE_OPTIONS)
  • a wrapper script that customizes the runtime startup behavior of a Lambda function

Modifying the runtime environment with internal AWS Lambda extensions

Internal extensions allow you to modify the runtime environment of your Lambda functions. This can be done in two ways:

Language-specific environment variables

Lambda supports configuration-only ways to enable code to be pre-loaded during function initialization through the following language-specific environment variables:

• JAVA_TOOL_OPTIONS - this environment variable allows you to set additional command-line variables in Lambda. For instance you can specify the initialization of tools, specifically the launching of native or Java programming language agents using the agentlib or javaagent options.
  • As an example you can custom garbage collector behavior by setting JAVA_TOOL_OPTIONS to -XX:+UseParallelGC. See AWS docs on JAVA_TOOL_OPTIONS for more details.
• NODE_OPTIONS - Lambda supports this environment variable - all available options would be a bit of an overkill for a blogpost, but you can find all of them in Node.js documentation.
  • Speaking of documentation - AWS docs say that "On Node.js 10x and above, Lambda supports this environment variable". Which is technically true but as of this writing the lowest available Lambda runtime version of Node.js is Node.js 14. I took the liberty of sending feedback to AWS to update the docs.
  • Side note: Node.js 20 Lambda was announced yesterday!
• DOTNET_STARTUP_HOOKS - to quote the docs: "on .NET Core 3.1 and above, this environment variable specifies a path to an assembly (dll) that Lambda can use.".

Wrapper scripts

You can create a wrapper script to customize the runtime startup behaviour of your Lambda function which allows you to set configuration parameters that cannot be set through language specific environment variables described above.

Bear in mind that invocations may fail if the wrapper script does not successfully start the runtime process, so tread carefully. Consider not modifying the wrapper script on a Friday.

When you use a wrapper script for your function, Lambda starts the runtime using your script. Lambda sends to your script the path to the interpreter and all of the original arguments for the standard runtime startup.

In order to specify the script set the value of AWS_LAMBDA_EXEC_WRAPPER environment variable to the path of the script. The script must be executable. Here's an example of a wrapper script for Python

  #!/bin/bash

  # the path to the interpreter and all of the originally intended arguments
  args=("$@")

  # the extra options to pass to the interpreter
  extra_args=("-X" "importtime")

  # insert the extra options
  args=("${args[@]:0:$#-1}" "${extra_args[@]}" "${args[@]: -1}")

  # start the runtime with the extra options
  exec "${args[@]}"

BTW, if you'd like to learn more about AWS Lambda environment variables, consider checking out Guide to default AWS Lambda environment variables.