DEV Community

Cover image for Dynamically start Docker Compose Services with a simple Makefile
Med Marrouchi
Med Marrouchi

Posted on

Dynamically start Docker Compose Services with a simple Makefile

#docker #devops #nginx #bash

Efficiently Managing Docker Compose Services with a Dynamic Makefile

As developers, we often find ourselves juggling multiple services when working on projects, especially when using Docker and Docker Compose. Sometimes, the services required in a production environment aren't necessary for local development. Other times, we might need to toggle specific services on or off based on the task at hand. Managing this can become cumbersome, but with a cleverly crafted Makefile, we can simplify the process significantly.

In this post, I'll walk you through a Makefile that dynamically manages Docker Compose files and services, allowing you to switch between environments and toggle services effortlessly.

The Challenge

Image description

When working with Docker Compose, you might have multiple services defined across different YAML files. For example:

  • docker-compose.yml (the main Compose file)
  • docker-compose.dev.yml (development-specific configurations)
  • docker-compose.prod.yml (production-specific configurations)
  • Additional service files like docker-compose.nginx.yml, docker-compose.smtp4dev.yml, etc.

Managing these files manually can be error-prone and time-consuming. You might need to include or exclude certain services or switch between development and production environments. Without automation, this can lead to complex docker-compose commands and potential mistakes.

The Solution: A Dynamic Makefile

The Makefile below addresses these challenges by:

  • Allowing you to toggle services on or off.
  • Dynamically including Docker Compose files based on enabled services.
  • Providing commands to start, stop, and destroy containers in different environments.
  • Ensuring that your .env file is correctly set up.

The Makefile 

# Makefile
FOLDER := ./docker

# The services that can be toggled
SERVICES := nginx smtp4dev

# Function to dynamically add Docker Compose files based on enabled services
define compose_files
  $(foreach service,$(SERVICES),$(if $($(shell echo $(service) | tr a-z A-Z)), -f $(FOLDER)/docker-compose.$(service).yml))
endef

# Function to dynamically add Docker Compose dev files based on enabled services and file existence
define compose_dev_files
  $(foreach service,$(SERVICES), \
  $(if $($(shell echo $(service) | tr a-z A-Z)), \
    $(if $(shell [ -f $(FOLDER)/docker-compose.$(service).dev.yml ] && echo yes), -f $(FOLDER)/docker-compose.$(service).dev.yml)))
endef

# Function to dynamically add Docker Compose prod files based on enabled services and file existence
define compose_prod_files
  $(foreach service,$(SERVICES), \
  $(if $($(shell echo $(service) | tr a-z A-Z)), \
    $(if $(shell [ -f $(FOLDER)/docker-compose.$(service).prod.yml ] && echo yes), -f $(FOLDER)/docker-compose.$(service).prod.yml)))
endef

# Ensure .env file exists and matches .env.example
check-env:
    @if [ ! -f "$(FOLDER)/.env" ]; then \
        echo "Error: .env file does not exist. Creating one now from .env.example ..."; \
        cp $(FOLDER)/.env.example $(FOLDER)/.env; \
    fi
    @echo "Checking .env file for missing variables..."
    @awk -F '=' 'NR==FNR {a[$$1]; next} !($$1 in a) {print "Missing env var: " $$1}' $(FOLDER)/.env $(FOLDER)/.env.example

init:
    cp $(FOLDER)/.env.example $(FOLDER)/.env

# Start command: runs docker-compose with the main file and any additional service files
start: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml $(call compose_files) up -d

# Dev command: runs docker-compose with the main file, dev file, and any additional service dev files (if they exist)
dev: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) up -d

# Start-prod command: runs docker-compose with the main file, prod file, and any additional service prod files (if they exist)
start-prod: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.prod.yml $(call compose_files) $(call compose_prod_files) up -d

# Stop command: stops the running containers
stop:
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) $(call compose_prod_files) down

# Destroy command: stops the running containers and removes the volumes
destroy:
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) $(call compose_prod_files) down -v
Enter fullscreen mode Exit fullscreen mode

Breaking Down the Makefile

Let's delve into how this Makefile works and how you can leverage it in your projects.

Variables and Services 

FOLDER := ./docker

# The services that can be toggled
SERVICES := nginx smtp4dev
Enter fullscreen mode Exit fullscreen mode
  • FOLDER: Specifies the directory where your Docker Compose files reside.
  • SERVICES: A list of services that you might want to toggle on or off.

Dynamic Functions

compose_files 

define compose_files
  $(foreach service,$(SERVICES),$(if $($(shell echo $(service) | tr a-z A-Z)), -f $(FOLDER)/docker-compose.$(service).yml))
endef
Enter fullscreen mode Exit fullscreen mode

This function:

  • Iterates over each service in SERVICES.
  • Checks if an environment variable corresponding to the uppercase service name is set (e.g., NGINX, SMTP4DEV).
  • If the variable is set, it includes the corresponding Docker Compose file.

compose_dev_files and compose_prod_files

These functions are similar to compose_files but specifically target development and production files, respectively. They also check if the files exist before including them.

Environment Check 

check-env:
    @if [ ! -f "$(FOLDER)/.env" ]; then \
        echo "Error: .env file does not exist. Creating one now from .env.example ..."; \
        cp $(FOLDER)/.env.example $(FOLDER)/.env; \
    fi
    @echo "Checking .env file for missing variables..."
    @awk -F '=' 'NR==FNR {a[$$1]; next} !($$1 in a) {print "Missing env var: " $$1}' $(FOLDER)/.env $(FOLDER)/.env.example
Enter fullscreen mode Exit fullscreen mode

This target ensures that your .env file exists and contains all the necessary variables as defined in .env.example.

Make Commands

start 

start: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml $(call compose_files) up -d
Enter fullscreen mode Exit fullscreen mode

Starts the containers using the main Docker Compose file and any additional service files based on the toggled services.

dev 

dev: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) up -d
Enter fullscreen mode Exit fullscreen mode

Starts the containers in development mode, including development-specific configurations and service files.

start-prod 

start-prod: check-env
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.prod.yml $(call compose_files) $(call compose_prod_files) up -d
Enter fullscreen mode Exit fullscreen mode

Starts the containers in production mode, including production-specific configurations and service files.

stop and destroy 

stop:
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) $(call compose_prod_files) down

destroy:
    @docker compose -f $(FOLDER)/docker-compose.yml -f $(FOLDER)/docker-compose.dev.yml $(call compose_files) $(call compose_dev_files) $(call compose_prod_files) down -v
Enter fullscreen mode Exit fullscreen mode
  • stop: Stops all running containers.
  • destroy: Stops all running containers and removes associated volumes.

How to Use This Makefile

Toggling Services

To enable or disable services, set environment variables corresponding to the uppercase service names.

For example, to enable nginx and disable smtp4dev:

make dev NGINX=1
Enter fullscreen mode Exit fullscreen mode

Alternatively, you can set these variables directly in your shell or include them in your .env file.

Switching Environments

  • Development: Use make dev to start services with development configurations.
  • Production: Use make start-prod to start services with production configurations.
  • Default: Use make start to start services with the default configurations.

Examples

Starting Services in Development Mode with nginx Enabled 

make dev NGINX=1
Enter fullscreen mode Exit fullscreen mode

Stopping All Services 

make stop
Enter fullscreen mode Exit fullscreen mode

Destroying All Services and Volumes 

make destroy
Enter fullscreen mode Exit fullscreen mode

Running Database Migrations 

make migrate-up
Enter fullscreen mode Exit fullscreen mode

Benefits of This Approach

  • Flexibility: Easily toggle services on or off without modifying Docker Compose files.
  • Environment Management: Seamlessly switch between development and production environments.
  • Automation: Simplify complex docker-compose commands into easy-to-remember make commands.
  • Consistency: Ensure that all team members use the same commands and configurations.

Conclusion

Managing multiple Docker Compose files and services doesn't have to be a headache. By using a dynamic Makefile, you can streamline your workflow, reduce errors, and focus on what matters most—writing great code.

Feel free to adapt and extend this Makefile to suit your project's needs. Happy coding!

Note: Remember to adjust the SERVICES variable and other configurations to match your specific project setup.

Top comments (0)

Read next

jakepage91 profile image

Glasskube Launch week #2: The wrap up

Jake Page -

andreacavallo profile image

Real-Time Temperature Monitoring with Go & gRPC!

Andrea Cavallo -

singh_in_cloud profile image

Day 28: Diving Into Real-World DevOps Projects on AWS

Kanavsingh -

marko_milosavljevic_97169 profile image

Terraform Modules and Workspaces: Techniques for Modular and Scalable Infrastructure in AWS

Marko Milosavljevic -