How often are you deploying a Lambda container image, basically a Lambda running on a Docker image, for a platform that doesn't match your localhost platform? Often I deploy functions running on ARM rather than on X86_64, this is a personal preference and it doesn't come with any massive advantage (there are online some comparison), and although my laptop is ARM-based, the CI/CD server is not :/
This is true even for GitHub Workflows which run on X86_64, rather than ARM if you don't create your custom runner.
So if you have a piece of code like the following, and you try to deploy it via a CI/CD server which is X86_64 it will fail with the error
The error
Warning: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
exec /bin/sh: exec format error
The command '/bin/sh -c ...' returned a non-zero code: 1
The code
from aws_cdk import (
Stack,
aws_lambda,
aws_ecr_assets as ecr
)
from constructs import Construct
class LambdaMultiplatDemStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
my_function = aws_lambda.DockerImageFunction(
self,
"MyDifferentPlatformFn",
code=aws_lambda.DockerImageCode.from_image_asset(
"."
),
architecture=aws_lambda.Architecture.ARM_64
)
The Dockerfile
FROM public.ecr.aws/lambda/python:3.8
COPY requirements.txt .
RUN pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"
COPY my_py_app.py ${LAMBDA_TASK_ROOT}
CMD [ "my_py_app.handler" ]
This is true even for the simplest Docker image because Docker is pulling executable dependencies for your it that are not compatible with your execution platform. How to solve this issue? Thanks to Docker and CDK!
Let's start with Docker
Is not news that Docker offers the possibility to build images for a platform different than yours using buildx
, to find out more just visit the official documentation link.
So first thing first, we need to check if buildx
is installed on our machine; you can follow the official doc here, but if you want to simplify the process
- Run
docker buildx ls
to list all the builders on your machine, if the output presents something similar to what is below you should be good (see the various platform available)
[panilo@fedora ~]$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default * docker
default default running 23.0.1 linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64
- If the output doesn't present the platform you're interested in, you need to
- run the following command
docker run --privileged --rm tonistiigi/binfmt --install all
- run again the
docker buildx ls
command, this time the platforms available should be more
- run the following command
Now we can build images for other platforms, great!
Step 2: CDK
Just modify the stack to add a special platform
property in the aws_lambda.DockerImageCode.from_image_asset
definition
from aws_cdk import (
Stack,
aws_lambda,
aws_ecr_assets as ecr
)
from constructs import Construct
class LambdaMultiplatDemStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
my_function = aws_lambda.DockerImageFunction(
self,
"MyDifferentPlatformFn",
code=aws_lambda.DockerImageCode.from_image_asset(
".",
platform=ecr.Platform.LINUX_ARM64 # Magic Switch!
),
architecture=aws_lambda.Architecture.ARM_64
)
By adding this little parameter CDK will know to use Docker's buildx build
command and prepare the image for the specified platform.
We can now deploy anywhere, building anywhere :)
Please find a complete example here.
I hope this helped you,
as usual, any feedback is more than appreciated!
Cheers
Danilo
Post published also on Virtuability's website here.
Top comments (0)