Dear Maintainer, Please Keep a Changelog

Changelogs. An important document for every (open source) project. Many projects have one. Many projects also try to keep it up to date.

But there is also a significant number of projects which do not have a changelog or which changelog is useless for the consumer of the project.

In this post I would like to showcase the pain and stress a bad changelog can cause for the consumer of a project, how a good changelog can help adoption of a project and solutions on how to keep the changelog of your next project up-to-date automatically.

Heads up: This post is a bit of a rant from the view of a consumer. I maintain a couple of projects at work which depend on PHP and JavaScript packages which we regularly update. The lack of bad changelogs in some of the packages inspired me to write this text. But I also publish and maintain my own open source projects. So, I know both sides of the coin.

Changelog != Release Notes

I differentiate between "Changelog" and "Release Notes".

A changelog is a single file in the projects repository which contains all significant changes in a project; grouped by version.

Release notes are published on GitHub or on the projects website. A subset of all releases might be displayed on a single page, but ususally each release note is seperated on its own page.

The content of release notes can be the same as the changelog for the respective version – for small projects for example – but they can also vary drastically.

Larger projects usually write lengthy explanations of what has changed, the motivation behind new features, thank you notes to collaborators or how to upgrade your code in case breaking changes are introduced.

Tailwind CSS does a terrific job at both writing changelogs and release notes.

For major and minor releases they write elaborate release notes. They explain what new features have been added, show examples how they can be used or guide you through changes. (See release notes for v2.2.0 or v3.0.0-alpha.1)

Their changelog follows the "Keep A Changelog" format. At the top is an unreleased heading that points to the compare view between the latest release and the last commit on the default branch. The heading for each released version is accompanied by a link to a compare view, to see exactly what has changed. For each released version the changes are sorted into categories like "Added", "Changed", "Fixed" or "Removed". And each single change has a link to the corresponding pull request, if one wants to read more about the change. *chef's kiss*

Upgrading dependencies which take their release notes, changelog and documentation as serious as Tailwind CSS is a joy. It allows me to minimize the time needed to make the upgrade as the maintainer provides all the information.

The Pain of a Bad Changelog

At work I manage a couple of projects depending on PHP and NPM packages. Each month we let Dependabot update our dependencies by creating a pull request we can then later review.¹

And this review process for NPM dependencies is a bit of a mess.

Some NPM projects don't have a changelog or release notes at all. Upgrading these dependencies without looking through the recent commits is like taking a lottery ticket and hoping for the best. Does the dependency still work as expected? Is the code published to npmjs.com the same as the one on github.com? Will upgrading the package deploy malicious code to my website?

In these cases we check the repository and review the latest commits. If our build is green and nothing shady is happening in the repo, the pull requests gets merged. This workflow however could be improved by providing – you guessed it – a changelog and good release notes.

Another fun pull request review experience are packages that are located in mono repos.

The screenshot above shows, the pull request Dependabot created for us to upgrade @babel/core from 7.15.8 to 7.16.0. You can also see the release notes Dependaboot pulled in for the 7.16.0 release.

Did you spot the single mention of @babel/core and what has changed? Only one line of the release notes is related to the @babel/core dependency I'm trying to upgade. The rest is just noise.

(babel is just an example here; and one of the dependencies I recently had to review. I love babel and am grateful for the team behind it for creating it. Having the power to use new JavaScript features sooner before they are available in all browsers is very nice.)

You might notice that I only mention NPM packages here.

In my experience, the documentation quality of PHP packages is higher than for NPM/JavaScript packages. Maybe that's because PHP packages ususally have a larger "scope" than JavaScript packages? I don't know.

A Solution

I don't want to end this post without giving options on how to make changelogs and release notes better. However, I also don't have all the solutions. Especially the issue mentioned with packages in mono repos seems unsolveable to me; or at least it's not feasable for maintainers. (Nobody wants to maintain a separate changelog for each package in a mono repo)

One approach I often see to auto-generate release notes is with Conventional Commits.

Each commit is prefixed with a tag like feat: or chore:. CLI tools or other software then compile a changelog based on the commits made since the last release. I personally haven't used this approach yet. I would probably forget to add the tag to the commit message for the first couple of days 😓.

The method I'm currently using, is a combination of the release-drafter GitHub Action and labels on GitHub. release-drafter compiles a list of the changes based on merged pull requests. It can categorize the merged pull requests either by branch name, pull request title or label.

I've chosen the latter approach, as it's very easy to apply and update labels in case a pull request has been mis-categorized. I've written more about this method here.

My next proposal is to automate the update process of the CHANGELOG.md file. For this I've created a GitHub Action called changelog-updater.

The Action can be used in a workflow like below, which is triggered whenever a new release is published on GitHub. The Action will take the given release name and body text and will add it to the CHANGELOG.md file.

name: "Update Changelog"

on:
  release:
    types: [released]

jobs:
  update:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2
        with:
          ref: main

      - name: Update Changelog
        uses: stefanzweifel/changelog-updater-action@v1
        with:
          release-notes: ${{ github.event.release.body }}
          latest-version: ${{ github.event.release.name }}

      - name: Commit updated CHANGELOG
        uses: stefanzweifel/git-auto-commit-action@v4
        with:
          branch: main
          commit_message: Update CHANGELOG
          file_pattern: CHANGELOG.md

If you want to learn more why I've created this Action, you can read the introduction blog post. Want to learn how it works? Read this blog post.

Conclusion

I hope this post showed how important release notes and changelogs can be, how they can improve "consumer happiness" and that it's worth your time as a maintainer to make these documents as good as possible.

If you have thoughts on this topic, questions or general feedback reach out to me via email or Twitter.

1. We also let GitHub Action auto merge some pull requests ↩