Jack Kweyunga

Posted on Sep 13 • Edited on Sep 15

Traefik Proxy Guide: Configuring public Domain Names for Docker Containers

#docker #traefik #devops

Disclaimer

This article introduces Traefik, a modern reverse proxy and load balancer for deploying micro-services. We will cover its basics, key features, configuration, and integration with Docker. By the end, you should understand how to set up and use Traefik in your projects.

The final example in this article builds on concepts discussed in my previous article: How to Automate Deployment with Ansible and GitHub Actions. In that article, I explained how to streamline your deployment process using Ansible for configuration management and GitHub Actions for continuous integration and deployment. If you haven't read it yet, I highly recommend doing so to fully grasp the advanced section of this article. Understanding the automation techniques covered there will be crucial for implementing the final example effectively.

Introduction

Traefik is a modern open-source reverse proxy and load balancer. It’s built with simplicity and flexibility in mind. Traefik excels in a containerized setup, especially with micro-services.

(Image source: Traefik Documentation)

Key features of Traefik proxy

Traefik comes with amazing features, including:

Dynamic configurations
Automatic SSL/TLS
Load balancing
Middleware support
Integration with other platforms

Configuring Traefik

Traefik configurations are written in familiar syntax i.e TOML or YAML .

Traefik gives you flexibility on the type of configuration you are comfortable with. It support the following types of configurations;

Static configuration where every or some configurations are defined in a single file; traefik.yml
Dynamic configuration where different providers are supported and used to automatically detect or create Traefik configurations.

For example, with the Docker provider, configurations are defined as labels on Docker containers and automatically detected as Traefik configurations.

Other supported providers include Swarm, Nomad, Kubernetes, Consul, and many more.

Traefik & Docker

In this article, our focus is on Traefik’s Docker provider. We are going to install traefik as a docker container and the use it to proxy a python flask application ( also a docker container ).

A Demo project

For this first demo, I’ll explain a simple setup using docker-compose. In the next demo, we will integrate with GitHub Actions and Ansible.

Here is the project structure.

traefik-demo\
|-- docker-compose.yml
|-- traefik\
|   |-- certs\
|   |-- traefik.yml

To get started, we need to define a Traefik configuration file. Here is what it should look like:

traefik.yml

global:
  checkNewVersion: true
  sendAnonymousUsage: false  # true by default

# (Optional) Log information
# ---
# log:
#  level: ERROR  # DEBUG, INFO, WARNING, ERROR, CRITICAL
#   format: common  # common, json, logfmt
#   filePath: /var/log/traefik/traefik.log

# (Optional) Accesslog
# ---
# accesslog:
  # format: common  # common, json, logfmt
  # filePath: /var/log/traefik/access.log

# (Optional) Enable API and Dashboard
# ---
api:
  dashboard: true  # true by default
  insecure: false  # Don't do this in production!

# Entry Points configuration
# ---
entryPoints:
  web:
    address: :80
    # (Optional) Redirect to HTTPS
    # ---
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: :443

# Configure your CertificateResolver here...
# ---
certificatesResolvers:
  staging:
    acme:
      email: "{ EMAIL }"
      storage: /etc/traefik/certs/acme.json
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      tlsChallenge: {}
      httpChallenge:
        entryPoint: web

  production:
    acme:
      email: "{ EMAIL }"
      storage: /etc/traefik/certs/acme.json
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      tlsChallenge: {}
      httpChallenge:
        entryPoint: web

providers:
  docker:
    exposedByDefault: false  # Default is true
  file:
    # watch for dynamic configuration changes
    directory: /etc/traefik
    watch: true

For automatic SSL to work, make sure to add a valid email. Replace “{ EMAIL }” with your own email address.

Let’s see how the docker compose file would look like.

docker-compose.yml

# Run traefik webserver, portainer, watchtower 

services:

  traefik:
    container_name: "traefik"
    image: "traefik:v2.5"
    ports:
      - "80:80"
      - "443:443"
      # - "59808:8080" # Uncomment this to expose the traefik dashboard
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - traefik-ssl-certs:/ssl-certs
      - ./traefik:/etc/traefik
    networks:
      - traefik_network
    restart: always

  api:
    image: ghcr.io/jackkweyunga/auto-deploy-flask-to-server:main
    environment:
      - FLASK_ENV=production
      - DEBUG=${DEBUG}
    expose:
      - "5000"
    network:
      - traefik_network
    labels:
      - traefik.enable=true    
      - traefik.http.routers.api.rule=Host(`mydomain.com`)
      - traefik.http.routers.api.entrypoints=websecure
      - traefik.http.services.api.loadbalancer.server.port=5000
      - traefik.http.routers.api.tls.certresolver=production
    restart: always

networks:
  traefik_network:
    external: true

volumes:
  traefik-ssl-certs:

Notice that we have defined a traefik_network in the docker-compose file as external. We need to create this network before running the file. Every other Docker container that will be proxied by Traefik must join the traefik_network.

sudo docker network create traefik_network

Notice the labels we added to the API service and that it is also part of the traefik_network. The labels define Traefik configurations. Below is a detailed explanation of each label.

traefik.enable=true

Tells Traefik to proxy this container/service.
traefik.http.routers.api.rule=Host(mydomain.com)

Tells Traefik to route all traffic from mydomain.com to this container/service. Ensure proper DNS records are set up to point the domain name to the server running Traefik.
traefik.http.routers.api.entrypoints=websecure

Tells Traefik to use a secure entry point for this container/service, as defined in the traefik.yml configuration file. This means using port 443, the secure port.
traefik.http.services.api.loadbalancer.server.port=5000

Tells Traefik to direct traffic to port 5000 of the container.
traefik.http.routers.api.tls.certresolver=production

Tells Traefik to use the production Let's Encrypt endpoints when fetching SSL certificates. This is also referenced in the traefik.yml configuration file.

NOTE: For each service, the router name must be unique from router names in other services. The same rule applies to service names. The router name is the part that comes after the dot, like traefik.http.routers.{router name}. If router names of different services are the same, proxying will fail.

When the setup runs on the server, Traefik will listen on ports 80 and 443. Any traffic on port 80 will be redirected to the secure port 443. Also, when someone visits mydomain.com, a response from the Flask application will be returned.

Add traefik to your CI/CD workflow

Referring to the article How to Automate Deployment with Ansible and GitHub Actions, where we set up our CI/CD workflow with Ansible and GitHub Actions, we will now build on that and add the Traefik proxy. This will let us assign public domain names to our services.

Follow this GitHub repository: https://github.com/jackkweyunga/auto-deploy-flask-to-server

Traefik Ansible role

First, we add a Traefik Ansible role. In this role, we define the automation needed to configure and run Traefik on a remote server. Below is the file structure, which can be added to the structure mentioned in the previous article.

--- previous ---
.github\
    workflows\
        deploy-proxy.yml
ansible\
    proxy.yml
    traefik\
        tasks\
            main.yml
        templates\
            traefik.yml.jinja2

templates/traefik.yml.jinja2

global:
  checkNewVersion: true
  sendAnonymousUsage: false  # true by default

# (Optional) Log information
# ---
# log:
#  level: ERROR  # DEBUG, INFO, WARNING, ERROR, CRITICAL
#   format: common  # common, json, logfmt
#   filePath: /var/log/traefik/traefik.log

# (Optional) Accesslog
# ---
# accesslog:
  # format: common  # common, json, logfmt
  # filePath: /var/log/traefik/access.log

# (Optional) Enable API and Dashboard
# ---
api:
  dashboard: true  # true by default
  insecure: false  # Don't do this in production!

# Entry Points configuration
# ---
entryPoints:
  web:
    address: :80
    # (Optional) Redirect to HTTPS
    # ---
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: :443

# Configure your CertificateResolver here...
# ---
certificatesResolvers:
  staging:
    acme:
      email: {{ EMAIL }}
      storage: /etc/traefik/certs/acme.json
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      tlsChallenge: {}
      httpChallenge:
        entryPoint: web

  production:
    acme:
      email: {{ EMAIL }}
      storage: /etc/traefik/certs/acme.json
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      tlsChallenge: {}
      httpChallenge:
        entryPoint: web

serversTransport:
  insecureSkipVerify: true

providers:
  docker:
    exposedByDefault: false  # Default is true
  file:
    # watch for dynamic configuration changes
    directory: /etc/traefik
    watch: true

This Traefik configuration template allows us to pass variables to it while saving it on the remote server. In this case, an EMAIL will be passed.

tasks/main.yml

---

- name: Preparing required files and Directories in /etc/traefik
  become: true
  block:

    - name: Create directory
      file:
        path: /etc/traefik
        state: directory

    - name: Create directory2
      file:
        path: /etc/traefik/certs
        state: directory

    - name: Copy config file
      ansible.builtin.template:
        src: templates/traefik.yml.jinja2
        dest: /etc/traefik/traefik.yaml

- name: Configuring traefik
  become: true
  block:

    - name: Create ssl-certs Volume
      community.docker.docker_volume:
        name: traefik-ssl-certs

      register: v_output
      ignore_errors: true

    - name: Debug output
      ansible.builtin.debug:
        var: v_output

    - block:
      - name: Create traefik_network
        become: true
        community.docker.docker_network:
          name: traefik_network
        register: n_output
        ignore_errors: true

      - name: Debug output
        ansible.builtin.debug:
          var: n_output

      when: v_output

    - block:

      - name: Deploy Traefik
        community.docker.docker_container:
          name: traefik
          image: "traefik:v2.10"
          ports:
            - "80:80"
            - "443:443"
            - "59808:8080"
          volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - traefik-ssl-certs:/ssl-certs
            - /etc/traefik:/etc/traefik
          networks:
            - name: "traefik_network" # required. Name of the network to operate on.
          restart_policy: always
          labels:
            com.centurylinklabs.watchtower.enable: "false"
        register: d_output
        ignore_errors: true

      - name: Debug output
        ansible.builtin.debug:
          var: d_output

      when: n_output

In this task, where the magic happens, we create the required folders, files, Docker volumes, and Docker networks (traefik_network). Then, we use Ansible’s docker_container community plugin to run Traefik on the remote server(s). If you look closely, you'll notice the similarity to the docker-compose.yml we used in the first demo.

ansible/proxy.yml

---
- hosts: webservers

  vars_files:
    - secret

  roles:
    - traefik

  vars:

    EMAIL: "{{ lookup('ansible.builtin.env', 'EMAIL') }}"

This is the main Ansible playbook we’ll run to configure Traefik. Notice how it references the Traefik role. We also read the EMAIL variable from the environment variable.

Now that the Ansible configurations are ready, let's add a GitHub workflow to run Ansible on demand with GitHub Action runners.

Add the proxy GitHub Workflow

.github/workflows/deploy-proxy.yml

name: proxy

on:
  workflow_dispatch:
    inputs:
      REMOTE_USER:
        type: string
        description: 'Remote User'
        required: true
        default: 'ubuntu' # Edit here 
      EMAIL:
        type: string
        description: 'Email for fetching certs'
        required: true
        default: '<put a valid email here>' # Edit here
      HOME_DIR:
        type: string
        description: 'Home Directory'
        required: true
        default: '/home/ubuntu' # Edit here
      TARGET_HOST:
        description: 'Target Host'
        required: true
        default: "< ip / domain >" # Edit here

jobs:
   ansible:
    runs-on: ubuntu-latest
    env:
      EMAIL: "${{ inputs.EMAIL }}"

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Add SSH Keys
        run: |
          cat << EOF > ansible/devops-key
          ${{ secrets.SSH_DEVOPS_KEY_PRIVATE }}
          EOF

      - name: Update devops private key permissions
        run: |
          chmod 400 ansible/devops-key
      - name: Install Ansible
        run: |
          pip install ansible

      - name: Adding or Override Ansible inventory File
        run: |
          cat << EOF > ansible/inventory.ini
          [webservers]
          ${{ inputs.TARGET_HOST }}
          EOF

      - name: Adding or Override Ansible Config File
        run: |
          cat << EOF > ./ansible/ansible.cfg
          [defaults]
          ansible_python_interpreter='/usr/bin/python3'
          deprecation_warnings=False
          inventory=./inventory.ini
          remote_tmp="${{ inputs.HOME_DIR }}/.ansible/tmp"
          remote_user="${{ inputs.REMOTE_USER }}"
          host_key_checking=False
          private_key_file = ./devops-key
          retries=2
          EOF

      - name: Run main playbook
        run: |
          sh ansible/create-sudo-password-ansible-secret.sh ${{ secrets.SUDO_PASSWORD }}
          ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/proxy.yml --vault-password-file=ansible/vault.txt

Make sure to edit the EMAIL and TARGET_HOST inputs.

We can now edit the flask-api ansible role to include traefik configurations via container labels.

Update the flask-api Ansible role

Edit ansible/flask-api/files/docker-compose.yml


volumes:
  portainer-data:

# Added networks
networks:
  traefik_network:
    external: true

services:

  portainer:
    image: portainer/portainer-ce:alpine
    container_name: portainer
    command: -H unix:///var/run/docker.sock
    expose:
      - "9000"
    volumes:
      # Connect docker socket to portainer
      - "/var/run/docker.sock:/var/run/docker.sock"
      # Persist portainer data
      - "portainer_data:/data"
    restart: always
    # Added networks
    networks:
      - traefik_network
    # Added lables
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.portainer.rule=Host(`${PORTAINER_DOMAIN}`)"
      - "traefik.http.routers.portainer.entrypoints=websecure"
      - "traefik.http.routers.portainer.tls.certresolver=production"
      - "traefik.http.routers.portainer.tls=true"
      - "traefik.http.services.portainer.loadbalancer.server.port=9000"
      - "traefik.docker.network=traefik_network"


  watchtower:
    container_name: "watchtower"
    image: "docker.io/containrrr/watchtower"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      # To enable docker authentication, uncomment the line below.
      # You also need to make sure you are logged in to docker in the server
      # E.g by running: sudo docker login ghcr.io
      # - /root/.docker/config.json:/config.json:ro
    restart: always
    environment:
      TZ: Africa/Dar_es_Salaam
      WATCHTOWER_LIFECYCLE_HOOKS: "1" # Enable pre/post-update scripts
    command: --debug --cleanup --interval 30

  web:
    image: ghcr.io/jackkweyunga/auto-deploy-flask-to-server:main
    environment:
      - FLASK_ENV=production
      - DEBUG=${DEBUG}
    expose:
      - "5000"
    restart: always
    # Added networks
    networks:
      - traefik_network
    # Added labels
    labels:    
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true    
      - traefik.http.routers.api.rule=Host(`${FLASK_API_DOMAIN}`)
      - traefik.http.routers.api.entrypoints=websecure
      - traefik.http.services.api.loadbalancer.server.port=5000
      - traefik.http.routers.api.tls.certresolver=production

    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

    deploy:
      resources:
        limits:
          memory: "256m"
          cpus: "0.50"

Check all places with the comment #Added ... These sections are due to adding the Traefik proxy configurations.

Notice new variables:

FLASK_API_DOMAIN: The domain name of our Flask API, e.g., api.mydomain.com
PORTAINER_DOMAIN: The domain name of our Portainer instance, e.g., portainer.mydomain.com

As a result, we need to add these variables to ansible/deploy.yml so that we can read them from the environment in GitHub Actions runners.

Edit: ansible/deploy.yml

---
- hosts: webservers

  # an encrypted ansible secret file containing the sudo password
  vars_files:
    - secret

  roles:
    - services

  environment:
    DEBUG: "{{ lookup('ansible.builtin.env', 'DEBUG') }}"
    # Added new variables
    FLASK_API_DOMAIN: "{{ lookup('ansible.builtin.env', 'FLASK_API_DOMAIN') }}"
    PORTAINER_DOMAIN: "{{ lookup('ansible.builtin.env', 'PORTAINER_DOMAIN') }}"

Update the GitHub workflow for deployment

Finally, we edit the deploy.yml GitHub actions workflow file to include the domains

name: ansible-deploy

on:
  workflow_dispatch:
    inputs:
      REMOTE_USER:
        type: string
        description: 'Remote User'
        required: true
        default: 'ubuntu'
      HOME_DIR:
        type: string
        description: 'Home Directory'
        required: true
        default: '/home/ubuntu'
      TARGET_HOST:
        description: 'Target Host'
        required: true
        default: "example.com" # Change this to your server IP or Domain

jobs:
   ansible:
    runs-on: ubuntu-latest
    env:
      DEBUG: 0
      # Added new variables
      FLASK_API_DOMAIN: "api.mydomain.com" # Add your domain
      PORTAINER_DOMAIN: "portainer.mydomain.com" # Add your domain
    steps:

      - name: Checkout
        uses: actions/checkout@v2

      - name: Add SSH Keys
        run: |
          cat << EOF > ansible/devops-key
          ${{ secrets.SSH_DEVOPS_KEY_PRIVATE }}
          EOF

      - name: Update devops private key permissions
        run: |
          chmod 400 ansible/devops-key

      - name: Install Ansible
        run: |
          pip install ansible

      - name: Adding or Override Ansible inventory File
        run: |
          cat << EOF > ansible/inventory.ini
          [webservers]
          ${{ inputs.TARGET_HOST }}
          EOF

      - name: Adding or Override Ansible Config File
        run: |
          cat << EOF > ./ansible/ansible.cfg
          [defaults]
          ansible_python_interpreter='/usr/bin/python3'
          deprecation_warnings=False
          inventory=./inventory.ini
          remote_tmp="${{ inputs.HOME_DIR }}/.ansible/tmp"
          remote_user="${{ inputs.REMOTE_USER }}"
          host_key_checking=False
          private_key_file = ./devops-key
          retries=2
          EOF

      - name: Run deploy playbook
        run: |
          sh ansible/create-sudo-password-ansible-secret.sh ${{ secrets.SUDO_PASSWORD }}
          ANSIBLE_CONFIG=ansible/ansible.cfg ansible-playbook ansible/deploy.yml --vault-password-file=ansible/vault.txt

Make sure to add working domain, that have DNS records directing them to the target server.

Operation

To install traefik on a remote server, run the proxy GitHub workflow. After that run the deploy GitHub workflow to update your infrastructure with the newly added traefik configurations and Domain names.

After both runs are successful, you can now visit your domains securely (with SSL). If SSL has not activated yet, give it some time and monitor Traefik’s logs in Portainer in case there is an issue.

Happy Configuring !

Conclusion

If you have reached this point, it is evident that Traefik significantly enhances your workflow. The Traefik proxy is an outstanding project that integrates seamlessly with containers and can be easily incorporated into your CI/CD pipeline.

With the addition of Traefik, our setup is nearly complete. Stay tuned for my next article.

Seeking expert guidance in Ops, DevOps, or DevSecOps? I provide customized consultancy services for personal projects, small teams, and organizations. Whether you require assistance in optimizing operations, improving your CI/CD pipelines, or implementing strong security practices, I am here to support you. Let's collaborate to elevate your projects. Contact me today | LinkedIn | GitHub

DEV Community