In an era where software deployment speed and security are paramount, automating critical tasks in the development pipeline is essential. GitHub Actions offers a robust platform to achieve this automation seamlessly.
In this article, we'll walk you through the creation of a GitHub Actions workflow that focuses on automating the patching and signing of container images using a CNCF sandbox project Copacetic.
Use copa-action to automate patching
Copacetic is a command-line application that uses reports from vulnerability scanners, such as trivy, to directly patch containers images.
It's able to do this by parsing the vulnerability reports generated by the scanners and creating additional layers on top of the container image that includes patches to the CVEs identified by the scanner.
For my Microsoft Build talk, I had to setup a shell task in the workflow to run the copa CLI, but luckily for you an official action now exists. Here's how you set it up.
# .github/workflows/patch.yml
on:
push:
branches:
- copaGitOps
jobs:
patch:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# provide relevant list of images to scan on each run
images: ['s3cexampleacr.azurecr.io/azure-voting-app-rust:v0.1.0-alpha']
steps:
- name: Login to Azure
id: login
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Login to Azure Container Registry
run: |
az acr login --name ${{ vars.ACR_NAME }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3.0.0
- name: Generate Trivy Report
uses: aquasecurity/trivy-action@69cbbc0cbbf6a2b0bab8dcf0e9f2d7ead08e87e4
with:
scan-type: 'image'
format: 'json'
output: 'report.json'
ignore-unfixed: true
vuln-type: 'os'
image-ref: ${{ matrix.images }}
- name: Check Vuln Count
id: vuln_count
run: |
report_file="report.json"
vuln_count=$(jq '.Results | length' "$report_file")
echo "vuln_count=$vuln_count" >> $GITHUB_OUTPUT
- name: Extract Patched Tag
id: extract_tag
run: |
imageName=$(echo ${{ matrix.images }} | cut -d ':' -f1)
current_tag=$(echo ${{ matrix.images }} | cut -d ':' -f2)
if [[ $current_tag == *-[0-9] ]]; then
numeric_tag=$(echo "$current_tag" | awk -F'-' '{print $NF}')
non_numeric_tag=$(echo "$current_tag" | sed "s#-$numeric_tag##g")
incremented_tag=$((numeric_tag+1))
new_tag="$non_numeric_tag-$incremented_tag"
else
new_tag="$current_tag-1"
fi
echo "patched_tag=$new_tag" >> $GITHUB_OUTPUT
echo "imageName=$imageName" >> $GITHUB_OUTPUT
- name: Copa Action
if: steps.vuln_count.outputs.vuln_count != '0'
id: copa
uses: project-copacetic/copa-action@v1.0.0
with:
image: ${{ matrix.images }}
image-report: 'report.json'
patched-tag: ${{ steps.extract_tag.outputs.patched_tag }}
buildkit-version: 'v0.11.6'
# optional, default is latest
copa-version: '0.3.0'
- name: Docker Push Patched Image
id: push
if: steps.login.conclusion == 'success'
run: |
# docker push ${{ steps.copa.outputs.patched-image }}
echo "DIGEST=$(docker push ${{ steps.copa.outputs.patched-image }} | grep -oE 'sha256:[a-f0-9]{64}')" >> $GITHUB_OUTPUT
There's a lot going on here, so let's walk through the steps in the job.
The workflow uses a matrix strategy with a list of images to generate a new job per image in the array of images.
A private Azure Container Registry is used the workflow authenticates to azure and then runs the az acr login command to populate the docker login credentials. Which allows you to use docker commands to interact with the registry.
The Trivy action is used to generate a list of vulnerabilities that exist for the targeted container image and outputs that to a report.json
file to be used for patching by Copacetic.
A few lines of Bash are used to populate a variable that contains the number of vulnerabilities found.
Extract Patch Tag is another shell task that uses for awksedfu to determine what the container image patched tag should be following the format tag-1
, with -1
indicating the image has been patched.
Copa action installs and runs the Copa CLI to ingest the reports.json
and patch the remote container image, which results in a local version of the image that's been patched.
- Once copa is done and a newly patched image has been created locally, the
docker push
command is used to push the image to the container registry, but it also grabs the digest from the image.
Curious what the digest is used for? Keep reading to find out. :)
Signed patched container images with Notation
Notation is another CNCF project that just hit version v1.0.0 in Aug. 202 and it allows you digitally sign artifacts. Those signatures become stamps of approval for those digital artifacts letting you know you can trust them to run in your environment.
And if you're using admission controllers on your cluster with projects like Ratify, those signatures are even more important because without them your containers won't be allowed to deploy on the cluster.
At any rate, adding the notary-actions to your GitHub Action Workflow is fairly straight forward:
- name: Setup Notation
if: steps.push.conclusion == 'success'
uses: notaryproject/notation-action/setup@v1
with:
version: "1.0.0"
- name: Notation Sign
if: steps.push.conclusion == 'success'
uses: notaryproject/notation-action/sign@v1
with:
plugin_name: azure-kv
plugin_url: https://github.com/Azure/notation-azure-kv/releases/download/v1.0.1/notation-azure-kv_1.0.1_linux_amd64.tar.gz
plugin_checksum: f8a75d9234db90069d9eb5660e5374820edf36d710bd063f4ef81e7063d3810b
key_id: ${{ vars.KEY_ID }}
target_artifact_reference: ${{ steps.extract_tag.outputs.imageName }}@${{ steps.push.outputs.DIGEST }}
signature_format: cose
plugin_config: |-
name=${{ vars.CERT_NAME }}
self_signed=false
notation-action/setup
installs the notation CLI on the GitHub action runner at the specified version.
notation-action/sign' optionally installs a plugin, in this case, the azure-kv plugin. It then takes the necessary arguments as parameters and then passes them to the
notation sign` command that digitally signs the target artifact (container image)
Test the Workflow
If you've followed along with each tutorial in this series, you have the Flux resources for the Azure Voting app deployed along with Flux resources for the image automation to detect when new container images are pushed to the container registry. And now, you have a workflow that patches the Azure Voting app container images.
So, now the fun begins. Once you commit and push the .github.com/workflows/patch.yml
file to the repository it will patch the Azure voting container image and push it to the container registry. Once that image is pushed the Flux image automation will detect that a new version of the image exists. When that happens the image automation will update the kustomization.yaml
with the newer tag, which will in turn trigger the Flux resources monitor your application's repo to redeploy the application to the Azure Kubernetes Cluster.
Run the following commands & watch the magic happen:
git add . && git commit -m 'Add copa patching workflow' && git push
Top comments (0)