Proposal for a Sunset Policy for a GitHub Action

I am the current maintainer of a GitHub Action for checking spelling in files in a GitHub repository. The action "GitHub Spellcheck Action" (rojopolis/spellcheck-github-actions) has grown in popularity and is used in several GitHub repositories, meaning that changes to the software and it's availability has become significant factors in it's life cycle. At the same time, it is based on open and free infrastructure and maintenance, so I decided it was time to introduce a sunset policy.

In this post I will try to outline the rationale for the sunset policy. If you do not care for the rationale, only for the proposal, just skip to "Proposal for a Sunset Policy" below.

The GitHub Spellcheck Action is based on a Docker image. Once in a while I delete images, which have become_stale_. Earlier I have just done so, but it has always felt like this was not the proper approach - so after giving it some thought, I came to the conclusion that I needed to come up with a proposal for a sunset policy. These are the thoughts and ideas, shaping the rationale for the proposal.

I have had a few bad releases, one in particular jumps to mind, even though people were very nice, it struck me that there are actually people depending on the software I maintain and if it breaks or goes away, it will influence them and their work.

Some statistics on the repository:

• 83 stars
• 32 forks
• 13 contributors - primarily casual contributors
• 440+ dependents - a few of them are my forks for PRs for bumping used version of this action, more on this later

This underlines that a bad release or an artifact going away can cause a lot of frustration, I am just not scratching my own itch with this work, like so many of my other projects.

As stated I am not the original author of this repository I am just the current maintainer. I had some ideas for improvements since I was a heavy user, making a lot of spelling mistakes and using the action for about 20+ repositories for primarily public available specifications. When GitHub actions came to the scene I was very happy to adopt their use and of course interested in feedback for my builds.

As I took over the maintenance I wanted to move the action to rely more on the Docker part to speed up execution time, by bringing down build time, since the Docker image was build with every run. First Docker repository founded release was version 0.5.0, the latest release being 0.23.2. Build feedback is important, but it also has to come as fast as possible and using a prebuilt Docker image instead of building it on the fly, seemed like a sensible approach.

Statistics on the releases:

• Total 28 releases
• 4 release non-prebuilt Docker images (0.1.0-0.4.0)
• 24 releases using prebuilt Docker images

This does however introduce several other challenges:

• The artefacts becoming stale
• Maintenance of versioned images

IMHO the lowered execution time, makes is viable, since the administration/maintenance is not as time consuming and it is pretty much build once, run many times.

All of the Docker images are served from DockerHub. As new images are released and uploaded, older images are showing a decline in use, indicated by lack of downloads/pulls.

The images are currently deleted manually and this is where the sunset policy will play an important role, since deleting an image for a particular version could render a given live configuration useless. I do recommend Dependabot for automatic maintenance and Renovate could also be an alternative, but not all users make use of these tools.

I do code searches and create PRs for repositories using older versions. So I guess in addition to the above mechanisms, you could just wait for me to create a PR at least for your public repository.

Statistics on open PRs by me:

• About 5 open PRs for updates to action configurations
• 1 is using version 0.11.0
• 1 is using version 0.13.0
• 2 is using version 0.14.0
• 1 is using version 0.16.0

There is still much potential work to be done.

If I do some more code searching (I am using SourceGraph's "Code Search", see my blog post about this for details), You can get some interesting statistics on what versions are used:

There are plenty of repositories, which could do with a PR and a hint on how to use either Dependabot or Renovate. Luckily it seems as of the usage is following along with the releases and new versions are picked up and take lead.

If we look at the Docker images pulled from DockerHub, We can see that all of the images are being pulled, but some have not been pulled for some time and others are picking up when being introduced.

zsh> hub-tool tag ls jonasbn/github-action-spellcheck --format json | jq 'sort_by(.LastPulled)| reverse' | jq -S --raw-output '.[] | [.Name, .LastPulled] | @tsv'
jonasbn/github-action-spellcheck:0.23.0 2022-05-07T08:10:01.439973Z
jonasbn/github-action-spellcheck:0.23.1 2022-05-07T04:08:52.108901Z
jonasbn/github-action-spellcheck:0.23.2 2022-05-07T04:08:52.108901Z
jonasbn/github-action-spellcheck:latest 2022-05-07T04:08:52.108901Z
jonasbn/github-action-spellcheck:0.21.1 2022-05-06T22:37:26.587984Z
jonasbn/github-action-spellcheck:0.20.0 2022-05-06T17:32:42.68943Z
jonasbn/github-action-spellcheck:0.22.1 2022-05-04T08:11:38.567402Z
jonasbn/github-action-spellcheck:0.22.0 2022-05-04T07:34:23.267566Z
jonasbn/github-action-spellcheck:0.13.0 2022-04-29T17:04:41.597812Z
jonasbn/github-action-spellcheck:0.14.0 2022-04-29T17:04:41.597812Z
jonasbn/github-action-spellcheck:0.17.0 2022-03-26T16:53:51.628996Z
jonasbn/github-action-spellcheck:0.21.0 2022-01-25T07:18:17.937951Z
jonasbn/github-action-spellcheck:0.19.0 2022-01-24T17:19:15.623017Z
jonasbn/github-action-spellcheck:0.18.0 2021-12-18T19:03:53.941591Z
jonasbn/github-action-spellcheck:0.16.0 2021-10-14T19:05:33.111348Z

What is interesting is however that some of the older images are still being actively used, which is perfectly okay. But as a maintainer I have a few key interests:

• I want users to adopt newer and more contemporary versions, since they hold more bug fixes and are generally more stable
• I do not want to support old versions, if somebody reports an error and it cannot be reproduced, or does not seem relevant in the latest version, an update will be recommended as the remedy/fix
• The documentation follows the latest version, so the user might have an interest in being on the latest version, with the most covering, correct and exact documentation

As a maintainer I have limited time, limited resources, but so does the users, so I want to be able to support the users most effectively and at the same time, not coerce them into using the latest version. The action is not their primary interest, they are working on something else and they are just making use of this action to assist with their work and easing their workload so they can focus on their software and users.

So a sunset policy has to act within the frames outlined above.

The life-cycle of the action can be divided into two sections:

• beta, indicated by semantic version numbers 0.X.X
• stable, indicated by major numbers being updated X.X.X, where the first digit is higher than zero

The two different states do not differ that much, but it is worth taking into account.

This action is still in beta, the step to make a 1.0.0 release might be closing in, this is however not the goal of this policy, which is to outline the sunset policy, a policy which should of course encompass future versions and this stabilization.

Currently users are pin-pointing exact versions. In order for these to be available for a relevant time frame, the proposal is for releases to be available for 365 days (a year), that would give the users a year to update to a newer version.

name: Spellcheck Action
on: push

jobs:
  build:
    name: Spellcheck
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - uses: rojopolis/spellcheck-github-actions@0.23.2
      name: Spellcheck

Example lifted from til repository

Looking at the usage on DockerHub shows that the proposed 365 days, seem to fit the usage well.

The current focus is on the beta versions, where releases, might be frequent, irregular and unpredictable. During the beta the idea is to have each release live for 365 days (a year), no matter how many releases are made.

When the beta is over the release schedule and sunset policy is expected to change. Releases are expected to become less frequent and the requirement for availability will point to keeping release around will extend beyond the 365 days. A release announcement should communicate the time frame for availability for earlier releases. Major releases could also be kept around for longer if required and for supporting the users, even when backwards compatibility cannot be supported.

The figure outlines that all Docker images prior to version 0.13.0 should be removed from Docker image repository at DockerHub, which they have been. 0.13.0 and 0.14.0 have passed it's sunset date so it will be removed at some point. As you can see both release were recently used, meaning GitHub repositories are presumably still pointing to these exact releases and a PR to nudge could be necessary. The challenge with 0.13.0 is not so big, whereas the sunsetting of version 0.14.0 requires more work, since the impact will be higher.

Introduction of Canonical Versions

A proposal for release tagging could be to introduce canonical versions, so for the beta (0.X.X), the canonical version is named: v0 and for a major release like 1.0.0 the canonical version would be: v1.

These tag would act like latest, which points to the newest image, which also refers to the latest exact version like 0.2.3. These versions would not be announced to the GitHub Marketplace, since they are indirect and would therefor not be picked up by Dependabot or Renovate or at least we have no interest in them being picked up.

The challenges with the canonical versions however are:

• They are indirect updates, meaning they rely on the latest release in a given series
• They circumvent the possible review phase provided via a direct update, handled by the regular process and normal toolchain (PRs, Dependabot, Renovate)

One of the goal behind tools like Renovate and Dependabot is the prevention of toolchain attacks.

This raises several concerns:

• Do the users of canonical versions trust the maintainers of the action?
• Could the action do something malicious?

The action is a repository like any other, to it too could be targeted, the action currently just downloads a repository, analyzes it and reports back to the user. Could it be replaced with something doing something unintended/malicious - yes, possibly.

Even for my own repositories, I review all PRs, even the ones from Dependabot and Renovate and I am not sure if canonical versions are a good idea.

The hope I do have is that with a stable release, the release frequency will drop, meaning the necessity for a release scheme relying on canonical versions it not required.

For now I will be trying out the canonical versions for some of my repositories and I will not document their use in the official documentation, meaning adoption might happen, but it will be kept to a minimum, so possible deprecation will have the least impact.

Proposal for a Sunset Policy

This is the basic sunset policy

• Exact versions (X.X.X) are available for 365 days
• Canonical versions (vX) will be available forever

Exact versions during the beta (0.X.X) are expected to be available for 365 days. When the first stable version is made (1.0.0), the 365 day time frame might be revisited, depending on the release cycles.

• The canonical version v0, will always point to the latest beta release (0.X.X)
• The canonical version v1, will always point to the latest release in the 1.X.X series
• v2 to the latest stable release in the 2.X.X series and so forth

Canonical versions (vX) are expected to be available forever. If and when several major versions become available (X.0.0), these might be announced to be EOL.

• Exact versions are made available via the GitHub marketplace and updates can be handled directly via Dependabot, Renovate or manually
• Canonical versions are not explicitly made to the GitHub Marketplace and maintenance will be handled indirectly and automatically

This is the policy I will evaluate for use for "GitHub Spellcheck Action", policy might change if deemed necessary. Especially the use of canonical versions, will have to be scrutinized. so please regard it as experimental and the proposal as a proposal.