Hi! If you read my last post, you already know how to host your static pages on Azure, but among us there are people who don't like to use the "big companies" solution and prefer to host things themselves under their own roof. Here is a solution for them.
This solution uses two open source software. One is Minio which is a S3 compatible storage made with GO language and 2nd is s3www which is simple server also made with GO which uses S3 storage as storage for static pages.
Oh, and I also use Traefik which is a reverse proxy server. It will also be responsible for generating Let's Encrypt certificates. This last one can be skipped if you want to use whatever proxy you already have, but for me this was the easiest to configure.
Prerequisites
As prerequisite you will need to install Docker and docker compose for which links to guides i mention bellow.
You can freely skip this one if you already have docker installed on you computer (or server) and go to the next step.
Setup Traefik
First set up the reverse proxy (Traefik mentioned above). This will route traffic to your Docker services and create certificates for the secure HTTP protocol using Let's Encrypt.
Add the following content to your docker-compose.yml
somewhere on your server e.g. ~/docker/traefik
.
version: '2'
services:
proxy:
image: traefik
command:
# --configFile=/traefik.yml
- "--api.dashboard=true"
- "--api.insecure=true"
- "--log.level=DEBUG"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.le.acme.httpchallenge=true"
- "--certificatesresolvers.le.acme.httpchallenge.entrypoint=web"
#- "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.le.acme.email=<your@email.tld>" # change to your email used for generating new Let's encrypt ceritificates
- "--certificatesresolvers.le.acme.storage=/letsencrypt/acme.json" # here are stored certificates informations
- "--accesslog=true"
restart: unless-stopped
networks:
- frontend
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./letsencrypt:/letsencrypt
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=web"
- "traefik.http.routers.traefik.rule=Host(`traefik.your-domain.tld`)" # Change this to your domain
- "traefik.http.middlewares.traefik-websecure-redirect.redirectscheme.scheme=websecure"
- "traefik.http.routers.traefik.middlewares=traefik-websecure-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=websecure"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik.your-domain.tld`)" # Change this to your domain
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=le"
- "traefik.http.routers.traefik-secure.service=api@internal"
- "traefik.http.services.traefik.loadbalancer.server.port=8080"
networks:
frontend:
external: true
To start it run following command
cd ~/docker/traefik
docker-compose up -d
Cool, from now on you've got reverse proxy running for all your services. Open http://traefik.your-domain.tld:8080
to see dashboard.
If you wish, you can disable public access to the Dashboard by setting a firewall rule to only allow connections from IP addresses you know. (Don't use this if your external IP address is dynamic)
Another way is to comment the following line- "8080:8080"
in docker-compose.yml file.
Setup Minio
Next you will need to configure Minio, as you will need a place to put your website files. So, create a new folder ~/docker/minio
that will contain your server configuration and add the following content to your docker-compose.yml
.
version: '3.7'
networks:
frontend:
external: true
services:
minio:
image: minio/minio:RELEASE.2020-08-07T01-23-07Z
restart: unless-stopped
networks:
- frontend
volumes:
- ./data:/data
environment:
MINIO_ACCESS_KEY: <YOUR-ACCESS-KEY>
MINIO_SECRET_KEY: <YOUR-SECRET-KEY>
command: server /data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
labels:
- "traefik.enable=true"
- "traefik.http.routers.minio.entrypoints=web"
- "traefik.http.routers.minio.rule=Host(`s3.your-domain.tld`)"
- "traefik.http.middlewares.minio-websecure-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.minio.middlewares=minio-websecure-redirect"
- "traefik.http.routers.minio-secure.entrypoints=websecure"
- "traefik.http.routers.minio-secure.rule=Host(`s3.your-domain.tld`)"
- "traefik.http.routers.minio-secure.tls=true"
- "traefik.http.routers.minio-secure.tls.certresolver=le"
- "traefik.http.routers.minio.service=minio"
- "traefik.http.services.minio.loadbalancer.server.port=9000"
# Don't need to use when your container use only one network but it is very important when you have multiple networks with container
- "traefik.docker.network=frontend"
Before you can start it you have to change some rows there.
- Update your secret and access keys
environment:
MINIO_ACCESS_KEY: <YOUR-ACCESS-KEY>
MINIO_SECRET_KEY: <YOUR-SECRET-KEY>
- Change address to your domain in labels
- "traefik.http.routers.minio.rule=Host(`s3.your-domain.tld`)"
- "traefik.http.routers.minio-secure.rule=Host(`s3.your-domain.tld`)"
this will tell the Traefik on which domain your service will run and then he can requsest for new certificate.
All the magic is in labels section
labels:
# Enable traefik for service
- "traefik.enable=true"
# Listen on HTTP :80
- "traefik.http.routers.minio.entrypoints=web"
# Domain for HTTP
- "traefik.http.routers.minio.rule=Host(`s3.your-domain.tld`)"
# Redirect to HTTPS
- "traefik.http.middlewares.minio-websecure-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.minio.middlewares=minio-websecure-redirect"
# Setting for HTTPS entrypoint
- "traefik.http.routers.minio-secure.entrypoints=websecure"
- "traefik.http.routers.minio-secure.rule=Host(`s3.your-domain.tld`)"
- "traefik.http.routers.minio-secure.tls=true"
# Here is configuration for Cert resolver - We are using Lets Encrypt
- "traefik.http.routers.minio-secure.tls.certresolver=le"
# Which service and port it using
- "traefik.http.routers.minio.service=minio"
- "traefik.http.services.minio.loadbalancer.server.port=9000"
# Network for running docker services
- "traefik.docker.network=frontend"
You copy this labels to your next services and change it based on you needs.
Start minio
cd ~/docker/minio
docker-compose up -d
Navigate to https://s3.your-domain.tld
and you will se Minio's login page. To login use your ACCESS-KEY and SECRET-KEY. In web UI you can do simple things as create new buckets, upload files or set permissions. The default user (the one which is created with first start) is admin, so it have access (R/W) to all buckets you create. It's ok if you using it only for myself, but its not wise but wne you want multiple users? So let me show you how can you create new user ang give it access only to those buckets you want.
Download Minio Client
First of all you will need Minio Client. So download it
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc --help
On Windows you can donwload it from here https://dl.min.io/client/mc/release/windows-amd64/mc.exe.
Login to server
To login to your server use following command
mc config host add mystorage https://s3.your-domain.tld <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api s3v4
Create user
Let's create our user which will have acces to our bucket with page
mc admin user add home <user-name-or-key> <super-strong-password>
You can create user with any name you want. If you want to login there over web maybe it is better to give it name you can remember, since i want use this user for application i generating for both of them string for example name have 24 characters and password 48. As i said you can use whatever suits your needs better.
Create bucket
Create bucket where you will store your static site files.
mc mb mystorage/<change-this-to-your-bucket-name>
Create bucket policy
Next create policy which will be assigned to user
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:ListAllMyBuckets"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<change-this-to-your-bucket-name>/*"
],
"Sid": "Public"
}
]
}
Upload policy to server
mc admin policy add mystorage <policy-name> <policy-name>.json
You can name your fiel with policy to any name you want. I just naming it with same name as my policy.
Apply policy to user
mc admin policy set mystorage "iso" user=<your-user-name>
This is last thing on our minio server. So we have
- Minio server
- Bucket for our page
- User with policy to Read and Write to this bucket.
Setup s3www
To server static site from minio you will need server, because you cant tell to minio server to use index files (in time i write this i cant find any information about it). In this example i show you how to use s3www.
To server webpage from minio is easy as
s3www -endpoint "https://s3.amazonaws.com" -accessKey "accessKey" \
-secretKey "secretKey" -bucket "mysite"
But we will need to run it in docker and tell to traefik what it have to do.
Here is docker-compose file i use to hos my sites:
version: "2"
networks:
frontend:
external: true
services:
s3www:
image: maymeow/s3www:latest
command:
- "-endpoint=https:/<change-to-your-s3-address>"
- "-accessKey=<change-this-to-user-name>"
- "-secretKey=<change-this-to-password>"
- "-bucket=<change-this-to-bucket-name>"
- "-address=0.0.0.0:8080"
restart: unless-stopped
networks:
- frontend
# ports:
# - "3000:3000"
labels:
- "traefik.enable=true"
- "traefik.http.routers.s3www.entrypoints=web"
- "traefik.http.routers.s3www.rule=Host(`<chage-this-to-your-page-address>`)"
- "traefik.http.middlewares.s3www-websecure-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.s3www.middlewares=gitea-websecure-redirect"
- "traefik.http.routers.s3www-secure.entrypoints=websecure"
- "traefik.http.routers.s3www-secure.rule=Host(`<chage-this-to-your-page-address>`)"
- "traefik.http.routers.s3www-secure.tls=true"
- "traefik.http.routers.s3www-secure.tls.certresolver=le"
- "traefik.http.routers.s3www.service=gitea"
- "traefik.http.services.s3www.loadbalancer.server.port=8080"
- "traefik.docker.network=frontend"
In <chage-this-to-your-page-address>
use just page domain without https://
or http://
.
Copy your site to minio
Everything is set so last thig you have to do is to copy your page data to your minio bucket:
cd /path/to/your/site
mc cp -r . cdn/<change-this-to-buckent-name>
Bonus
I showed you how to upload site manually and as bonus here is script for gitlab runner to uppload contend vi CI/CD.
deploy to azure:
stage: deploy
image: maymeow/minio-cli
cache:
key: themaymeow-com-build
paths:
- public
policy: pull
dependencies:
- pages
script:
- mc config host add cdn <change-your-s3-address-> $CDN_ACCESS_KEY $CDN_SECRET_KEY --api s3v4
- cd public
- mc cp -r . cdn/<change-this-to-buckent-name>
# Just for visualy check if there are files
- mc ls cdn/<change-this-to-buckent-name>
only:
- master
tags:
- docker
- digitalocean
Don't forget to create variables CDN_ACCESS_KEY
with minio user name and CDN_SECRET_KEY
with minio user password.
If you have more questions feel free to contact me.
Originally posted on https://themaymeow.com
Top comments (3)
please use something like grammerly.com, missing important words in every 3rd sentence feels like reading something that a 5 year old has written, just a friendly advice
Funny you would say something about 5y old and then misspell in your 1st line "grammerly" :).
LOL