I host the blog of a community project on my personal server. It's a static website generated with Jekyll, and the contents are versionned in a git repository.
In this blog post, I explain how I made the blog automatically update on every new commit. It's the same idea as Github Pages, but for self-hosting.
TL;DR
- create a script to clone the git repository, then build and deploy the website
- publish a webhooks endpoint on the server
- configure the GitHub repository to send a webhook on every new commit
Deploy script
The website is hosted on a Debian server, and the files are served from the /var/www/atlas.tecnologia.bo/ directory. To update it from command line, first install the Jekyll requirements, then create and launch the bash script:
$ /opt/deploy_atlas.tecnologia.bo.sh
that contains:
#!/usr/bin/env bash
# Clone git repository
rm -rf /tmp/atlas.tecnologia.bo
git clone https://github.com/RipeAtlasBolivia/atlas.tecnologia.bo.git /tmp/atlas.tecnologia.bo
# Generate blog
cd /tmp/atlas.tecnologia.bo
bundle install
bundle exec jekyll build --source /tmp/atlas.tecnologia.bo/ --destination /var/www/atlas.tecnologia.bo/
# Clean
rm -rf /tmp/atlas.tecnologia.bo
Webhook daemon
Instead of manually launching the script, let a webhook daemon do the job every time it receives an HTTP request on an endpoint.
First install the webhook package:
$ sudo apt install webhook
Verify the service is running:
$ sudo service webhook status
● webhook.service - Small server for creating HTTP endpoints (hooks)
Loaded: loaded (/etc/systemd/system/webhook.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-08-12 08:25:24 UTC; 2 weeks 0 days ago
Docs: https://github.com/adnanh/webhook/
Main PID: 419 (webhook)
Tasks: 6 (limit: 4697)
Memory: 4.8M
CGroup: /system.slice/webhook.service
└─419 /usr/bin/webhook -verbose -hooks /etc/webhook.conf
We configure the webhook daemon, creating the file /etc/webhook.conf with the following content (see the webhook documentation for more details):
[
{
"id": "atlas",
"execute-command": "/opt/deploy_atlas.tecnologia.bo.sh",
"command-working-directory": "/tmp/",
"trigger-rule": {
"match": {
"type": "payload-hash-sha1",
"secret": "xxxxxxxxx",
"parameter": {
"source": "header",
"name": "X-Hub-Signature"
}
}
}
}
]
This creates a new hook called atlas, that will be triggered when a GET or POST request is received on http://localhost:9000/hooks/atlas. When triggered, it will check that a valid X-Hub-Signature HTTP header has been provided, and then launch the /opt/deploy_atlas.tecnologia.bo.sh script from the /tmp working directory.
Try it:
- compute the
X-Hub-Signaturecorresponding to thexxxxxxxxxsecret
$ echo -n "" | openssl sha1 -hmac "xxxxxxxxx"
(stdin)= d46d87941e6f285be78ff0f1c8ea32620577b9ef
- request the endpoint:
$ curl -X POST -H "X-Hub-Signature: sha1=d46d87941e6f285be78ff0f1c8ea32620577b9ef" -v http://localhost:9000/hooks/atlas
...
> POST /hooks/atlas HTTP/1.1
> Host: localhost:9000
> User-Agent: curl/7.64.0
> Accept: */*
> X-Hub-Signature: sha1=d46d87941e6f285be78ff0f1c8ea32620577b9ef
>
< HTTP/1.1 200 OK
< Date: Mon, 26 Aug 2019 13:45:09 GMT
< Content-Length: 0
<
- check the logs:
$ grep webhook /var/log/syslog
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 Started POST /hooks/atlas
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 [d2ae30] incoming HTTP request from [::1]:58800
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 [d2ae30] atlas got matched
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 [d2ae30] atlas hook triggered successfully
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 Completed 200 OK in 670.302µs
Aug 26 14:06:40 webhook[10552]: [webhook] 2019/08/26 14:06:40 [d2ae30] executing /opt/deploy_atlas.tecnologia.bo.sh (/opt/deploy_atlas.tecnologia.bo.sh) with arguments ["/opt/deploy_atlas.tecnologia.bo.sh"] and environment [] using /tmp/ as cwd
Aug 26 14:06:46 webhook[10552]: [webhook] 2019/08/26 14:06:46 [d2ae30] command output: Cloning into '/tmp/atlas.tecnologia.bo'...
...
Aug 26 14:06:46 webhook[10552]: Bundle complete! 4 Gemfile dependencies, 24 gems now installed.
Aug 26 14:06:46 webhook[10552]: Use `bundle info [gemname]` to see where a bundled gem is installed.
Aug 26 14:06:46 webhook[10552]: Configuration file: /tmp/atlas.tecnologia.bo/_config.yml
Aug 26 14:06:46 webhook[10552]: Source: /tmp/atlas.tecnologia.bo/
Aug 26 14:06:46 webhook[10552]: Destination: /var/www/atlas.tecnologia.bo/
Aug 26 14:06:46 webhook[10552]: Incremental build: disabled. Enable with --incremental
Aug 26 14:06:46 webhook[10552]: Generating...
Aug 26 14:06:46 webhook[10552]: Jekyll Feed: Generating feed for posts
Aug 26 14:06:46 webhook[10552]: done in 0.45 seconds.
Aug 26 14:06:46 webhook[10552]: Auto-regeneration: disabled. Use --watch to enable.
Aug 26 14:06:46 webhook[10552]: [webhook] 2019/08/26 14:06:46 [d2ae30] finished handling atlas
Note: the 200 HTTP code only means the hook has been found for the atlas identifier and the X-Hub-Signature is valid. It says nothing about the deploy script success or failure.
In order to publish the endpoint from a Apache webserver, configure a reverse proxy in your Apache configuration:
ProxyPass /webhook/ http://localhost:9000/hooks/
ProxyPassReverse /webhook/ http://localhost:9000/hooks/
GitHub configuration
Finally, in order to trigger a webhook on every new push, go to the GitHub project settings, then "Webhooks", and create a new webhook with the following parameters:
- Payload URL:
https://mydomain/webhook/atlas - Content type:
application/json - Secret:
xxxxxxxxx - Which events would you like to trigger this webhook?:
Just the push event.
Alternative configuration with GitLab
If your git repository is hosted on a GitLab instance, you must change the headers check in the /etc/webhook.conf file:
[
{
"id": "atlas",
"execute-command": "/opt/deploy_atlas.tecnologia.bo.sh",
"command-working-directory": "/tmp/",
"trigger-rule": {
"match": {
"type": "value",
"value": "xxxxxxxxx",
"parameter": {
"source": "header",
"name": "X-Gitlab-Token"
}
}
}
}
]
Note: the X-Gitlab-Token header provided in the request will contain the value xxxxxxxxx, not its HMAC as it occurs with GitHub. So, to simulate the request, adapt the curl command to:
$ curl -X POST -H "X-Gitlab-Token: xxxxxxxxx" -v http://localhost:9000/hooks/atlas
Then, the GitLab project configuration is similar to GitHub: go to the project "Settings", "Integrations", and add a webhook with:
- URL:
https://mydomain/webhook/atlas - Secret Token:
xxxxxxxxx - Trigger: [x] Push events
Top comments (7)
Another question ;)
How im able to use the same webhook services for diferent request
Im try something like:
[
{
"id": "atlas",
"execute-command": "/scripts/myscript1.sh",
"command-working-directory": "/tmp/",
"trigger-rule": {
"match": {
"type": "value",
"value": "secrect1",
"parameter": {
"source": "header",
"name": "X-Gitlab-Token",
}
}
}
}
]
[
{
"id": "atlas2",
"execute-command": "/scripts/myscript1.sh",
"command-working-directory": "/tmp/",
"trigger-rule": {
"match": {
"type": "value",
"value": "secrect2",
"parameter": {
"source": "header",
"name": "X-Gitlab-Token",
}
}
}
}
]
Im try in diferent ways but all the time i get:
[webhook] 2019/12/17 00:06:22 Started POST /hooks/atlas2
[webhook] 2019/12/17 00:06:22 Completed 404 Not Found in 30.368µs
[webhook] 2019/12/17 00:06:28 Started POST /hooks/atlas
[webhook] 2019/12/17 00:06:28 Completed 404 Not Found in 34.086µs
If i use only one all works fine:
[
{
"id": "atlas",
"execute-command": "/scripts/myscript1.sh",
"command-working-directory": "/tmp/",
"trigger-rule": {
"match": {
"type": "value",
"value": "secrect1",
"parameter": {
"source": "header",
"name": "X-Gitlab-Token",
}
}
}
}
]
[webhook] 2019/12/17 00:02:37 Started POST /hooks/atlas
[webhook] 2019/12/17 00:02:37 Completed 200 OK in 164.625µs
I think it's because of your configuration JSON format: instead of
you should have
You could use a linter like jsonlint.com/?json= or prettier.io/ to check your configuration file.
Thanks....
Thats solve the issue..
Nice. Thanks for this.
But, is this secure?
The hook scripts in
execute-commandrun asrootwhen using weebhook as a service (systemd).Sure, you might want to edit
/etc/systemd/system/webhook.serviceto specify the user, for example a dedicatedwebhookuser created for these tasks only:Nice...
How i can debug the execution of the command...
Im not able to check why anything happends evens when i get a access sucessful...
You can access the logs produced by the command with:
I updated the blog post to add this information.
Note that you need access to the server to see these logs. From a client point of view, you will only see a 200 HTTP code if the command has been launched successfully, but this doesn't mean the command itself has worked as expected.