Intro
From the moment my friend Tomas recommended me Fly.io, I just in love with it for my side projects. Dubbed as the "Repacked Spirit of OG Heroku" - and from the past 6 months using I can testify to the truth of the statement!
Copy-pasted Dockerfiles taking over my project
This article focuses on how to re-use your custom Docker base across multiple Fly.io apps. My personal use case were deploying multiple batch jobs each having the same Python base but a few tweaks in the end for the entrypoint or config values. Making sure all 4 of those Dockerifles are the same was somewhat annoying to me so first I was like
TOOD(P1, devx): Have a common Docker base image".
FROM python:3.12-slim
WORKDIR /app/backend
# `git` is required by python package `supawee`
# `postgresql-dev` and `libffi-dev` are for psycopg[binary,pool] (this is always such a pain to install)
RUN apt-get update && apt-get install -y \
git \
libpq-dev \
# ... blah blah blah
The moment
But then a sub-package wanted to install tiktoken
which for performance reasons requires rustup
so now I had to copy-past that snippet to all my Dockerfiles and it takes like 1-2 mins to build I was aaargh if I only had one base image for my project I only have to build it once! So that's how it started.
Result
Pasting two of my Dockerfile
s to get you an idea of how re-using the common Dockerfile.base
can help you manage your deployment easier:
FROM registry.fly.io/web-scraping-batch-jobs-base-registry:latest
CMD ["python", "-m", "batch_jobs.scraper.daily_scrape"]
and second
FROM registry.fly.io/web-scraping-batch-jobs-base-registry:latest
ENV YOLO=true
CMD ["python", "-m", "batch_jobs.upsert_plugins"]
Solution
Well it took me a few hours to figure this out! :face-palm:
At the high level:
- build the base image locally
- push it to fly.io registry
- fly deploy with app-specific Dockerfile referencing FROM:
Resulting script
Here I employ a new style of blog writing, context-based explaining of each (hopefully) self-describing command.
#!/bin/bash
# TODO(P1, blog): Write down my experience building this for fly.io
# Usage deploy_batch_job.sh scraper/fly.toml
FLY_CONFIG_PATH=$1
# Set variables
BASE_DOCKERFILE="<your-path-to>/Dockerfile.base"
# Here `<my-project>-base-registry` is a dummy app on fly.io which is just used for storing base docker images.
# You can create such a dummy app by `fly apps create $REGISTRY_DUMMY_APP_NAME`
REGISTRY_DUMMY_APP_NAME="<my-project>-base-registry"
IMAGE_NAME="registry.fly.io/$REGISTRY_DUMMY_APP_NAME"
# we do --platform linux/amd64 to match the one fly.io builders have
# https://github.com/superfly/rchab/blob/8d37d90dc7d418660b50a10f288715fda4a00b5d/build.sh#L7
PLATFORM="linux/amd64"
echo "Logging into fly.io registry"
# Authentication successful. You can now tag and push images to registry.fly.io/{your-app}
fly auth docker
echo "Building batch jobs base image $DOCKERFILE ($PLATFORM) to {$IMAGE_NAME}"
# Build the image locally
docker build --platform $PLATFORM -t $IMAGE_NAME:latest -f $DOCKERFILE .
# Get the image ID (content hash), removing the "sha256:" prefix as a
IMAGE_ID=$(docker inspect --format='{{.Id}}' $IMAGE_NAME:latest | cut -d: -f2)
echo "Image ID: $IMAGE_ID (content hash)"
FULL_IMAGE_NAME="$IMAGE_NAME:$IMAGE_ID"
# Tag the image with its content hash
docker tag $IMAGE_NAME:latest $FULL_IMAGE_NAME
# Check if the image already exists in the registry
if docker manifest inspect $FULL_IMAGE_NAME > /dev/null 2>&1; then
echo "Image $FULL_IMAGE_NAME"
echo " -- already exists in the registry."
echo " -- Skipping push."
else
echo "Pushing new image $FULL_IMAGE_NAME to the registry."
docker push $IMAGE_NAME:$IMAGE_ID
docker push $IMAGE_NAME:latest
fi
echo "Deploying the batch job $FLY_CONFIG_PATH"
# To debug problems with Fly.io app builders you can find them at https://fly.io/dashboard/<your-organization-snake-case>/builders
# or with CLI fly logs -a fly-builder-<assigned-builder-name> (get it from logs)
fly deploy --config $FLY_CONFIG_PATH
Discussion
Was it worth it?
Well, I conclude that this effort to be net positive for humanity I had to share it! Cause truth to be told, continuing doing careful copy-pasting (a synonym to devops
from a devops person I really appreciate) would been more effective for me. But hey, we lazy engineers who want to automate our job right?
Also #lesscodelessbugs
Feature: If the image is same, do not push it
It took a few Claude Sonet (I am sick of ChatGPT for programming) I managed to setup a content-hash based push if doesn't yet exist approach to save some deploy time. Cause you know, if deploy takes over 30 seconds you tab-out to some stupid YouTube video or visit MK and then it takes 5mins to deploy.
Some gotchas
- There is the
--platform
flag to match your local build with thefly.io
build. Some versions of Docker Client (like Docker Desktop) had problems with that. I personally usecolima
which is also much more battery efficient.
Possible improvements
- Would be nice to build the
Dockerfile.base
using the fly.io workers; certainly possible with I guessfly.toml
. -
FROM <your-base-image>
takes some while even when both imae
Appendix
The full git commit setting this up in my side-project to see a
"production" version linked here on my github.
Top comments (0)