When working with Docker Compose, enabling caching can significantly speed up your builds by reusing layers from previous builds. This article will guide you through the process of setting up caching for Docker Compose, ensuring your development workflow is as efficient as possible.
Prerequisites
Before we start, make sure you have the following installed:
- Docker CLI
-
yqCLI (YAML processor) -
docker-compose.yamlfile
Step-by-Step Guide
Step 1: Check for Necessary Tools
First, we need to ensure that the required tools are installed on your system. Here’s a script that checks for Docker CLI, yq CLI, and the presence of the docker-compose.yaml file.
# Check for Docker CLI
if ! command -v docker &> /dev/null
then
echo "Docker CLI not found. Please install Docker."
exit 1
fi
# Check for yq CLI
if ! command -v yq &> /dev/null
then
echo "yq CLI not found. Please install yq."
exit 1
fi
# Check for docker-compose.yaml
if [ ! -f ./docker-compose.yaml ]; then
echo "docker-compose.yaml not found. Please ensure the file exists in the current directory."
exit 1
fi
Step 2: Set Up Preflight Script
The preflight script prepares the environment for caching. It sets up a temporary file path for the caching configuration.
# Set up necessary environment variables if not already set
: "${DOCKER_IMAGE:=your_default_docker_image}"
: "${DOCKER_REGISTRY:=your_default_docker_registry}"
: "${DOCKER_USER:=your_default_docker_user}"
: "${DOCKER_PASSWORD:=your_default_docker_password}"
# Preflight script for cacher-preflight
working_dir=$(dirname "$(realpath ./docker-compose.yaml)")
temp_file="$working_dir/docker-compose.caching.yaml"
export CACHE_COMPOSE_PATH=$temp_file
export DOCKER_BUILDKIT=1
Step 3: Set Up Cacher Script
The cacher script modifies the Docker Compose file to include cache settings for each service.
# Copy into cache path, and add caching
cp ./docker-compose.yaml $CACHE_COMPOSE_PATH
services=$(yq -r '.services | keys | .[]' ./docker-compose.yaml)
for service in $services; do
yq eval -i ".services.${service}.build.cache_from[0] = \"type=registry,ref=$DOCKER_IMAGE:${service}-buildcache\"" $CACHE_COMPOSE_PATH
yq eval -i ".services.${service}.build.cache_to[0] = \"type=registry,ref=$DOCKER_IMAGE:${service}-buildcache,mode=max\"" $CACHE_COMPOSE_PATH
done
Step 4: Build and Push Script
Finally, the build script builds and pushes the Docker images, utilizing the caching configurations set up in the previous steps.
# Build script for build
docker buildx create --use
# After using buildx, you will need to relogin into docker.
echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USER" --password-stdin $DOCKER_REGISTRY
docker compose -f $CACHE_COMPOSE_PATH build
docker compose -f $CACHE_COMPOSE_PATH push
Conclusion
With this I was able to speed up compilation in CI/CD greatly by utilizing remote caching. :)
Top comments (0)