π Intro
A few months ago, we have started to migrate our sourcecode CI & CD to Github.com.
Then, both DEV and OPS started to migrate source codes.
It appeared that OPS had to migrate a huge amount of Ansible related repos from an onPrem Gitlab instance to Github.com, and we wanted to take advantage of linting and code quality of the yamls.
OPS started to develop the migration process so it could be entirely code driven and tested.
What appeared was that we needed to manage that huge amount of code to :
- 1οΈβ£ lint (syntax validity, but for weirdnesses like key repetition and cosmetic problems such as lines length, trailing spaces, indentation, etc)
- 2οΈβ£ fix/implement lints feedbacks in a massive way (they were so huge that we could not imagine do that by hand)
OPS did choose to use the following approach :
As DEV & OPS are using yaml
everyday a -bit- lot more (k8s, CI configuration, Spring Boot configuration, ...), we started to
think about a common toolbox that we could embed in our CI pipeline, on Github.com.
What we needed to do was to :
- Integrate OPS sourcecode (DevOPS teamwork)
- Package it inside as a dedicated GH Action that everyone could add to their pipeline within a few lines of code
π οΈ Yaml Fixer
An important work has been done by OPS on yamlfixer so it could be used conveniently from the cli
:
β About
yamlfixer automates the fixing of problems reported by yamllint by parsing its output.
πΌοΈ Screenshot
Here'a screenshot of yamlfixer launched on yaml-test-suite :
π Installation
The easiest way to install yamlfixer
is from
pypi, as described
below.
π§ Linux install
python3 -m pip install yamlfixer-opt-nc
πͺ Windows install
python -m pip install yamlfixer-opt-nc
pipx
setup
For an optimal experience we recommand using pipx
.
To install :
pipx install yamlfixer-opt-nc
pipx list
To upgrade :
pipx upgrade yamlfixer-opt-nc
To uninstall :
pipx uninstall yamlfixer-opt-nc
π Usage
This software automatically fixes some errors and warnings reported by
yamllint
.
usage: yamlfixer [-h] [-v] [-b] [-B BACKUPSUFFIX] [-d] [-D DIFF_FILE] [-e EXTENSIONS] [-f]
[-F] [-l] [-N] [-n] [-r LEVEL] [-j | -p | -s] [-t TABSIZE]
[-c CONFIG_FILE | -C CONFIG_DATA]
[FILE_or_DIR [FILE_or_DIR ...]]
Fix formatting problems in YAML documents. If no file is specified, then reads
β¦Still this cli
needed a Python runtime to be used.
π³ Deliver as a Docker
image
Then we wanted to distribute it an easier way, which meaned :
package it as a Docker image.
At that point, DEVs took the relay and implemented Docker release on GH CI, and pushed images on DockerHub under optnc/yamlfixer :
π Deliver as a GH Action
Now, we could use the Docker image to implement the following Public GH Action :
opt-nc / yamlfixer-action
Github action that fixes some errors and warnings reported by yamllint and create a pull request with fixes.
β About
This github-action is based on the tool Yamlfixer
:
- Developed by
@tamere-allo-peter
- Integrated as a Github Action by
@mbarre
It automatically fixes some errors and warnings reported by yamllint
and creates a pull request that embeds the fixes.
π§° Usage
See action.yml
You need a Github account that is allowed to create pull request on the inspected repository.
Find below an example that allows to check yaml
files on each push with verbose mode :
name: Lint yaml files
on: [push]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout my app
uses: actions/checkout@v3
- name: Lint yaml files
uses: opt-nc/yamlfixer-action
with:
options: --summary
user: ${{secrets.my_user}}
token: ${{secrets.my_user_password}}
The github-action creates :
- A new branch named
yamlfixer/patch/$branch_name
- The pull request to be merged into the working branch
π Resources
Here are some useful resourcesβ¦
... that could be released on the Marketplace :
π€ Action usage
Here is the code (and... it's yaml
ππ₯π£) required to use the action in any workflow :
name: Lint yaml files
on: [push]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout my app
uses: actions/checkout@v2
- name: Lint yaml files
uses: opt-nc/yamlfixer-action
with:
yaml_file: .github/*.yml
options: --verbose
user: ${{secrets.my_user}}
token: ${{secrets.my_user_password}}
π Notice that the action creates a new branch and the pull request to be merged into the working branch so this code activity can be managed and reviewed as any other source code proposal.
π¦ Live demo
As a live video worths a thousand words : discover how the Github Action runs on a dedicated repository :
π· Examples in details
β Rejected PR because of lint failure
βοΈ Valid PR coming from GH Action
And code change review :
πͺ ROI
Now, both DEVs and OPs are using it to enhance yaml
quality proactively with the help of CI and yaml
quality issues are managed on our daily workflow on GH Issue.
π Also notice that all the teams using the GH Action have the same level of quality accross the whole organization.
π Acknowledgments π§βπ€βπ§
A lot of thanks to the team that made this work come true, all on our DEVOPS pipeline :
-
Jerome ALET as core
Yaml Fixer
developer & integrator - Michèle BARRE for Docker packaging and GH Action development
- Daniel SANTOS for its test on DockerHub publish and his continuous feedbacks on this new Gh Action
Top comments (0)