- Initial thoughts
- Prerequisites
- The GitLab pipeline code
- Integrated features
- Remove obsolete folders after concurrency problems
- Wrapping up
- Further reading
Initial thoughts
With GitLab Pages, you can publish static websites directly from a repository in GitLab. By default, we cannot have preview pages: if a job deploys the pages, this overwrites previous content, which disallows preview mode.
Some article on the internet show how to get around that with artifacts, knowing that GitLab can display artifacts. But this trick has disadvantages, mainly highly technical links, that have to be shared again after changes on the branch.
Some Github gist points into the right direction, but in a complex way and with too few side features.
In this article, we will get around the limitation by taking advantage of the cache mechanism, and be able to display per-branch content, with the side benefit of obfuscating the path to ephemeral branches content, if desired.
The solution can be broken down to these steps:
- Generate files for current branch
- Get previous branches generation from GitLab cache
- Merge and update cache
- Auto delete obsolete cache on branch deletion, using GitLab environments
Prerequisites
We assume you already have a way of generating your HTML static content, and just want to serve the files using GitLab Pages.
For the code to work, your cache must be centralized, either by using gitlab.com runners, by having a single runner, or by sharing caches between multiple private runners.
You need to accept a global cache by deactivating the GiLab option Use separate caches for protected branches in Settings -> CICD -> General Pipelines.
The GitLab pipeline code
workflow:
rules: # disable tag pipelines and duplicate MR pipelines
- if: $CI_COMMIT_BRANCH
variables:
MAIN_BRANCH_PATH: "."
EPHEMERAL_BRANCHES_PATH: preview # subpath to ephemeral branches content for preview
pages:
stage: build
image: alpine:3.18
cache:
key: gitlab-pages
paths: [public]
before_script:
# default available 'tree' app in alpine image does not work as intended
- apk add tree
# CURRENT_CONTENT_PATH is defined in rules, different between main branch and ephemeral branches
- mkdir -p public/$CURRENT_CONTENT_PATH && ls public/$CURRENT_CONTENT_PATH/..
- | # avoid deleting main branch content when cache has been erased
if [ "$CI_COMMIT_BRANCH" != "$CI_DEFAULT_BRANCH" ] && [ ! -f public/$MAIN_BRANCH_PATH/index.html ]; then
echo -e "💥\e[91;1m Unable to retrieve $CI_DEFAULT_BRANCH generated files from cache ; please regenerate $CI_DEFAULT_BRANCH files first\e[0m"
exit 1
fi
- rm -rf public/$CURRENT_CONTENT_PATH || true # remove last version of current branch
script:
- ./generate-my-html.sh --output public/$CURRENT_CONTENT_PATH || true # insert here your code that generates documentation
- cd public/$EPHEMERAL_BRANCHES_PATH
- tree -d -H '.' -L 1 --noreport --charset utf-8 -T "Versions" -o index.html # generate a root HTML listing all previews for easier access
environment:
name: pages/$CI_COMMIT_BRANCH
action: start
url: $CI_PAGES_URL/$CURRENT_CONTENT_PATH
on_stop: pages-clean-preview
rules:
# default branch is exposed at GitLab Pages root
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
variables:
CURRENT_CONTENT_PATH: $MAIN_BRANCH_PATH
# other (short-lived) branches generation are exposed in 'EPHEMERAL_BRANCHES_PATH/branch-name-sanitized' sub path
- variables:
CURRENT_CONTENT_PATH: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG
artifacts:
paths: [public]
expire_in: 1h
pages-clean-preview:
stage: build
image: alpine:3.18
cache:
key: gitlab-pages
paths: [public]
variables:
GIT_STRATEGY: none # git files not available after branch deletion
FOLDER_TO_DELETE: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG # an indirection to allow arbitrary deletion when launching this job
script:
- rm -rf public/$FOLDER_TO_DELETE
environment:
name: pages/$CI_COMMIT_BRANCH
action: stop
rules:
- if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
when: manual
allow_failure: true
Integrated features
The above code has below features:
-
maincontent is exposed on$CI_PAGES_URLand the path is configurable with$CURRENT_CONTENT_PATH - Per-branch preview content is exposed on
$CI_PAGES_URL/preview, with a homepage to easily navigate to branches content - Path to root preview folder is configurable with
$EPHEMERAL_BRANCHES_PATHvariable to hide preview content by obfuscation - Generated pages are associated with environments to take advantage of auto-cleaning on branch deletion
- To avoid disturbing already existing environments, pages environment are placed under a
pagesfolder - If
maincontent has not been generated in current cache, or if the cache has been deleted, an error is triggered, to avoid accidental deletion - Deletion job can be triggered manually with any cache path as input, to clean outdated data
- Code can safely be added to an existing project pipeline without causing trouble with already existing jobs
- The
workflow:rulescan be deleted if you already have your own, or updated to match your flow - The job must be named
pagesand the artifact must be apublicfolder to be deployed to GitLab Pages (or you can use the pages:publish keyword)
Remove obsolete folders after concurrency problems
If you experience concurrency problems leading to obsolete folders from deleted branches, you have multiple choices:
- Manually launch
pages-clean-previewwith a folder name - Manually clear the GitLab cache, regenerate content for default branch, and let other branches be updated on their future pipelines
- Use the resource_group keyword
- This may slow down your pipelines, waiting to acquire the lock
pages:
[...]
resource_group: avoid-cache-racing-conditions
pages-clean-preview:
[...]
resource_group: avoid-cache-racing-conditions
-
Delete obsolete folders using
git branchat the end ofpagesjobs:
variables: GIT_DEPTH: 0 # all branches are fetched (and known) script: [...] - apk add git - git branch -r - cd $EPHEMERAL_BRANCHES_PATH - | for folder in *; do exists=$(git rev-parse -q --verify "origin/$folder") || true if [ -z "$exists" ]; then rm -rf $folder echo "removed folder $folder, the branch does not exist anymore" fi done [...]
Wrapping up
Given the piece of yaml provided, using it in your pipeline, you should be able to share your GitLab Pages on a per-branch basis, on the path you want, while serving the stable content from the root context.
For any question or remark, please use below comment section 🤓.
If you need information to choose better runner architecture, you can read GitLab Runners topologies: pros and cons.
Illustrations generated locally by Automatic1111 using RevAnimated model with PiratePunkAI and Blindbox LoRA
Top comments (4)
In
pages-clean-previewjob, should it be:Thanks, nice catch ! I only tested when the commit branch name is kebab-case.
There should also be another fix, 'preview' is parameterized as 'EPHEMERAL_BRANCHES_PATH' :
I'm updating right away 🔧
In before_script we should be checking for ! -d public instead of ! -d public/$CI_DEFAULT_BRANCH right? Since we are putting the content of default branch in root of public directory.
You are right, another difference with my usual setup (I usually have the main branch generated files in a the common subfolders).
I'm even replacing
-d public/$CI_DEFAULT_BRANCHwith-f public/$MAIN_BRANCH_PATH/index.html, more solid proof that something has been generated !What do you think ?