Managing upstream security fixes in uselagoon docker images

This week, the Git project announced a maintenance release to patch two CVEs (CVE-2022-41903 and CVE-2022-23521).

I thought this would be a good opportunity to cover how the Lagoon team handles ensuring these updates make it through to our uselagoon-produced images.

The majority of our images are based on upstream images, created by the application, framework, or language maintainers.

Using The Node.js runtime as an example:

• Lagoon publishes a range of node images on Docker Hub (e.g. https://hub.docker.com/r/uselagoon/node-18/tags)
• This image is built using a Dockerfile
• We inherit the upstream image FROM node:18.13-alpine3.17 and then we add a number of customisations.
• This node image is just one of a range published by the Node.js team (https://hub.docker.com/_/node), and they also have the Dockerfile for their build available
• The node image itself is built from the upstream image FROM alpine:3.17 and customised to add the nodejs runtime and supporting libraries.
• The Dockerfile used to build the Alpine image is also available - this just unzips the relevant Alpine tarball into an empty "scratch" image and sets the shell

I can digress slightly to explain the image naming/tagging we use:

• uselagoon/node-18:22.12.0 is the latest release of our Node.js version 18 image (December 2022 - we use CalVer for our tag convention, and release monthly)
• node:18.13-alpine3.17 in the uselagoon image is an alias tag for the latest version 18.13 Node.js image, built using the Alpine Linux distribution, version 3.17. There are at least 10 aliases for this same image - in our case, we want to pin it to a specific minor release (18.13) using a specific Alpine version (3.17) - we could tighten or relax these, but we feel this gives us more control over automated updates
• alpine:3.17 in the Node.js image is the alias for the latest Alpine 3.17 release (currently 3.17.1). Alpine is minor version updated twice a year https://alpinelinux.org/posts/Alpine-3.17.0-released.html

I know this sounds really tortuous and convoluted, but bear with me!

Lagoon also publishes a version of our Node.js image, with additional tools used to "build" Node.js projects - we call it node-18-builder - see the Dockerfile. We add a range of packages to the standard image, as they are not included out of the box.

Because the Lagoon team maintains the images here, we are responsible for ensuring that any components added at this stage are secure, and have any available updates. The two packages I'm going to focus on here are curl and git, as they have some of the best documented vulnerability announcement and management strategies.

Alpine-based packages (apks) are managed in the Alpine package registry, and updates are usually tied to the Alpine Linux release cycle:
e.g. Git on Alpine 3.17 is currently at version 2.38.3-r1 - which is the correct version to address the above CVEs!

In the Git repository linked from the actual package there is also an APKBUILD file that outlines a bit more information - and from this, we can see that those CVEs were actually addressed in the previous 2.38.3-r0 release.

There may even be occasions where the Alpine Linux version of a package includes the CVE fix before the source package - you can see this clearly on curl on Alpine 3.16 - the 7.83.1 version of curl has been patched over 10 times now - https://curl.se/docs/vuln-7.83.1.html - the updated 7.87 version is only available via Alpine 3.17

Reading these dockerfiles, Git repositories and package logs is a bit of a "dark art", but rest assured that the Lagoon team secretly loves it, and it's all part of our role in ensuring the safety and security of the applications running on our platforms!