DEV Community

Webhook Relay profile image Karolis
Karolis for Webhook Relay

Posted on • Edited on

Docker Compose update on GitHub release webhook

#docker #automation #devops #git

Alt Text

Last year I wrote a blog post about combining several tools to automate simple NodeJS app updates on git push. Many users were solving similar problems by writing local web servers in Ruby, Python or PHP to receive webhooks and then do async processing. I am happy to announce that we finally decided to add this feature to the relay CLI. Now, to execute a bash script, you can:

relay forward --bucket my-bucket-name --relayer exec --command bash update.sh
Enter fullscreen mode Exit fullscreen mode

And to launch a Python app on webhook:

relay forward --bucket my-bucket-name --relayer exec --command python my-app.py
Enter fullscreen mode Exit fullscreen mode

This opens up some interesting posibilities to create pipelines that can react to pretty much anything that emits webhooks. In this article I will show you how to build a GitOps style pipeline that keeps a Docker Compose deployment in sync with a docker-compose.yaml hosted on a git repository.

Prerequisites

Repository with scripts that I used for this article can be found here: https://github.com/webhookrelay/docker-compose-update-on-git-push.

Step 1: Deploying containers through Docker Compose

First step is to do the initial deployment. We will create a simple dockerized Python application that you can find here that connects to Redis and deploy it:

version: '3'
services:
  web:
    image: "karolisr/python-counter:0.1.0"
    ports:
      - "5000:5000"
  redis:
    image: "redis:alpine"
Enter fullscreen mode Exit fullscreen mode

Step 2: Setting up updates on Github tag

Since we only want to update on git tags and not just any pushes, let's configure a webhook and analyse the payload.

To achieve that, let's first create a bucket with an internal output:

$ relay forward --bucket docker-compose-update-on-git-push http://localhost:4000
Forwarding:
https://my.webhookrelay.com/v1/webhooks/a956a9f7-2260-4bc2-a54b-3d896acf4206 -> http://localhost:4000
Starting webhook relay agent...
2019-08-28 23:14:41.773 INFO    using standard transport...
2019-08-28 23:14:41.928 INFO    webhook relay ready...  {"host": "my.webhookrelay.com:8080", "buckets": ["8e977e70-09a6-464c-ad30-855e1cd5d9f9"]}
Enter fullscreen mode Exit fullscreen mode

Here, bucket will be used later to subscribe to github requests while destination is just a mandatory argument that we don't have to use in this case.

Grab that https://my.webhookrelay.com/v1/webhooks/*** URL and go to your repository's settings -> webhooks section. Once there, set:

  • Payload URL to your unique https://my.webhookrelay.com/v1/webhooks/*** URL
  • Content type to application/json
  • Secret to a random secret name, for the sake of this example my secret will be 'webhooksecret'
  • Click Let me select individual events. and select Releases.

Now, go to your repository's releases page (for example https://github.com/webhookrelay/docker-compose-update-on-git-push/releases) and make a new release 1.0.0. Then, if you visit bucket details page or logs page - you should see webhook from Github. Open it and let's inspect the payload. It's quite lengthy but we should be able to see

 "action": "released",
Enter fullscreen mode Exit fullscreen mode

in the top. To ensure that we only react on these events, create a rule:

release filter

If you tag another release, you should see now that only one webhook was forwarder

rule stopped webhook

Step 3: Update script and starting relay background service

Our update script is:

#!/bin/bash

git pull
docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

It will pull the latest compose file and update containers. Now, let's update configuration in relay.yml file (access key & secret can be generated here):

version: v1
key: xxx     # your access key
secret: xxx  # your access secret
buckets:
- docker-compose-update-on-git-push # your bucket name where github webhooks are sent
relayer:
  type: exec
  command: bash
  commandArgs:
  - /full/path/to/docker-compose-update-on-git-push/update.sh # <-- should be full path to your update script
  timeout: 300
Enter fullscreen mode Exit fullscreen mode

To start the relay, run:

relay run -c relay.yml
Enter fullscreen mode Exit fullscreen mode

This will be running it through your terminal. For production use cases, please use background service mode. It will ensure that the daemon is launched on OS startup.

Let's try it out

Launch docker-compose:

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Check containers:

$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                    NAMES
26cd2219e18b        redis:alpine                    "docker-entrypoint.s…"   20 seconds ago      Up 3 seconds        6379/tcp                 docker-compose-update-on-git-push_redis_1
63c8cd1ae7bb        karolisr/python-counter:0.1.0   "flask run"              20 seconds ago      Up 18 seconds       0.0.0.0:5000->5000/tcp   docker-compose-update-on-git-push_web_1
$ curl http://localhost:5000
I have been seen 1 times.
$ curl http://localhost:5000
I have been seen 2 times.
Enter fullscreen mode Exit fullscreen mode

Next step would be to build a new image 0.2.0 and push it to the registry. Once it's available, we can update our github repository docker-compose.yml and make a new release. For the sake of this example, let's do this through the Github UI.

In a few seconds you should see a new container running:

$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                    NAMES
27b2542423ec        karolisr/python-counter:0.2.0   "flask run"              9 seconds ago       Up 7 seconds        0.0.0.0:5000->5000/tcp   docker-compose-update-on-git-push_web_1
26cd2219e18b        redis:alpine                    "docker-entrypoint.s…"   10 minutes ago      Up 9 minutes        6379/tcp                 docker-compose-update-on-git-push_redis_1
Enter fullscreen mode Exit fullscreen mode

Optional: Validating Github secret

Webhook Relay output rules can also validate Github signature:

webhook filter signature

This will ensure that only webhooks signed by Github will be processed.

Alternative solutions

I have somewhat successfully used these tools to update containers:

Both tools work if you have a simple setup that just needs to track latest tags. However, if you have multiple containers with semver (or any other versioning mechanism) - those solutions do not work. That's why it's useful to have an either .env or docker-compose.yml config with image tags committed into git and synchronized into the host that runs the services.

Conclusion

As with any code executed on your machine - you have to be careful when automating tasks. Webhook Relay will provide you with a unidirectional flow of webhooks into the machine. Your script/applications are on your machine and cannot be modified remotely through Webhook Relay. Coupled with authenticated webhook endpoints (you can configure it on a bucket level) or webhook payload checksum validation - you can build a secure update mechanism.

Originally released on https://webhookrelay.com/blog/2019/09/02/docker-compose-update-on-github-webhooks/

Top comments (0)

Read next

pcmagas profile image

How I can run elasticsearch locally for development using docker?

Dimitrios Desyllas -

sandr0p profile image

Code Reviews: Easing the pain

sandr0-p -

omkara18 profile image

Terraform Drift: Deep Dive

omkar shelke -

cicubeteam profile image

Kubernetes Horizontal Pod Autoscaler

CiCube -