Flawless Releases With Docker

This is a streamlined remix of my original Flask and
Heroku tutorial.

Docker containers are such a great way of packaging applications—once we get our app in Docker, we get a lot of benefits:

• ⚖️ Consistency: prod and dev are equal.
• 🚗 Portability: fewer dependencies with the underlying OS; the same image can be deployed on many cloud provider.
• 🗺 Divide and conquer: distribute services among different containers.
• ⚙️ Low overhead: near-native speeds.

However, Docker introduces a new variable to the equation: the code must be baked into an image, then correctly deployed. Also, testing can be more challenging. Here is where CI/CD can be of great value to us: by automating all those tasks and giving us a fast, reliable environment to work with.

Here’s the Plan

We already have a ready-to-use Python Flask demo. The app is a simple task manager, written for Flask; it uses MongoDB as the database backend.

The app is split in two: one container for the database and one for the web app. The CI/CD workflow has to:

1. Build the Docker images and push them to Docker Hub.
2. Test the app inside the container.
3. Deploy it to Heroku as a Docker Dyno.

We’ll be using the following tools:

• Heroku CLI.
• Semaphore CLI.

To get started, fork the semaphore-demo-python-flask repository and clone it. Then, add the project to Semaphore:

$ cd semaphore-demo-python-flask 
$ sem init

Create an empty application; app names are unique, so be creative 🐴.

$ heroku create bright-horses
Creating app... done, ⬢ bright-horses

The next thing is setting up the database. Here you can make a choice between two offerings:

• MongoLab: super easy to set up, integrates nicely with Heroku, and the sandbox offers 500MB free to use. You need to add a credit card in Heroku to use it, though. I’ll use this alternative.
• MongoDB Atlas: the free plan offers 500MB in a three server cluster, which is excellent for production. The setup, however, is more involved. I cover it extensively on the original tutorial

MongoLab can be installed in your app with a single command:

$ heroku addons:create mongolab:sandbox --app YOUR_APP_NAME

The Way Pipelines Work

A well designed Continuous Integration (CI) workflow will help us to:

• ⏲ Spend less time testing and deploying.
• 🤖 Get 100% automated testing.
• 😷 Avoid it-works-on-my-machine syndrome.

Shall we take a quick look at the CI pipeline? It can be found at .semaphore/semaphore.yml.

Pipelines begin with the agent and name. Agents are the machines. where our jobs run.

version: v1.0
name: Semaphore Python / Flask / Docker Example Pipeline
agent:
  machine:    
    type: e1-standard-2
    os_image: ubuntu1804

A Block defines actions for the pipeline. Each block has a single task, and each task can have one or more jobs. Jobs within a block run concurrently, each one in its own fully isolated machine. Once all jobs in a block complete, the next block begins.

The pipeline starts with the “Build” block:

1. secret imports the $DOCKER_* variables from the vault.
2. checkout clones GitHub repository.
3. Docker login is required for pushing the image to Docker Hub.
4. docker-compose builds the image…
5. …which is tagged, and finally pushed to the registry.

blocks:
  - name: Build
    task:
      secrets:
        - name: pyflask-semaphore
      jobs:
      - name: Docker build
        commands:
          - echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
          - checkout
          - docker-compose build
          - docker tag pyflasksemaphore:latest "$DOCKER_USERNAME"/pyflasksemaphore:$SEMAPHORE_WORKFLOW_ID
          - docker push "$DOCKER_USERNAME"/pyflasksemaphore:$SEMAPHORE_WORKFLOW_ID
          - docker images

The last block spins up the containers for integration testing. The prologue is executed before every job in a block. In this case, the prologue pulls the image and starts the app:

  - name: Run & Test Docker image
    task:
      secrets:
        - name: pyflask-semaphore
      prologue:
        commands:
          - echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
          - checkout
          - docker pull "$DOCKER_USERNAME"/pyflasksemaphore
          - docker-compose up -d
      jobs:
      - name: Check Running Images
        commands:
          - docker ps
      - name: Run Unit test
        commands:
          - docker exec -it semaphore-pyflask-docker_flasksemaphore_1 python -m unittest

The final section specifies how the workflow continues. At this point, we can chain multiple pipelines with promotions to create complex workflows. Promotions can be manual or automatic. In this case, we have a manual connection to the deployment pipeline:

promotions:
  - name: Deploy to Heroku
    pipeline_file: deploy-heroku.yml

The Deploy Pipeline

The last pipeline has the “Deploy to Heroku” block. Open the .semaphore/deploy-heroku.yml file; we have to make some changes before using it:

1. Remove the “atlas-mongodb” line in the secret section. We won’t be needing that variable.
2. Remove the “heroku set DB=$MONGODB_URI” line. Heroku integration will set its Mongo environment for us.
3. Set the HEROKU_APP variable to your app name.

This is the end result:

blocks:
  - name: Deploy to Heroku
    task:
      secrets:
        - name: pyflask-semaphore
        - name: heroku
      env_vars:
        - name: HEROKU_APP
          value: YOUR_APP_NAME_GOES_HERE
      jobs:
        - name: Deploy
          commands:
            - checkout
            - echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
            - docker pull "$DOCKER_USERNAME"/pyflasksemaphore:$SEMAPHORE_WORKFLOW_ID
            - heroku container:login
            - docker tag "$DOCKER_USERNAME"/pyflasksemaphore:$SEMAPHORE_WORKFLOW_ID registry.heroku.com/$HEROKU_APP/web
            - docker push registry.heroku.com/$HEROKU_APP/web
            - heroku stack:set container --app $HEROKU_APP
            - heroku container:release web --app $HEROKU_APP

Ready to Go

Create an authorization token on Heroku; this will allow Semaphore to connect with your account:

• Go to: Account Settings > Application > Create Authorization.

Copy the generated token and store it securely on Semaphore as a secret:

$ sem create secret heroku -e HEROKU_API_KEY=YOUR_API_TOKEN

Create a second secret to store your Docker Hub username and password:

$ sem create secret pyflask-semaphore \
    -e DOCKER_USERNAME=YOUR_DOCKER_HUB_USERNAME \
    -e DOCKER_PASSWORD=YOUR_DOCKER_HUB_PASSWORD

Push the changes to GitHub to get the CI/CD going:

$ git add .semaphore/*
$ git commit -m "deploy to Heroku"
$ git push origin master

The workflow starts automatically on every push:

Once you are satisfied, push the Promote button:

In a few seconds, the app should be online at http://APPNAME.herokuapp.com

The Real Hero-Ku Is You

We learned how to use Semaphore to create a pipeline that automates running the tests and the necessary build commands, as well as to deploy the application to Heroku.

Want to deliver continuously your applications made with Docker? Check out Semaphore’s Docker platform.

Did you find the post useful? Hit those ❤️ and 🦄, and leave a comment below.

Thanks for reading!