Automatically notify your team when you deploy new changes in a Python project

Photo from ThisisEngineeringPexels

Introduction

One of the things that I love about the Agile methodology is how it gives you the ability to make frequent tiny improvements to your project. An essential advantage of “being” Agile is deploying small changes frequently that add value to our customers or improve some internal processes.

In scrum, we would usually have a demo after every sprint in which we show new features/changes to other stakeholders(design, data, non-technical teammates etc.). However, in these demos, we rarely go into the minute details of the new changes (e.g. a column being renamed in a database table or the meaning of a column changing). These omissions could lead to problems with teams that depend on your work.

Imagine that you have a web team(that adds and maintains features on your web app) and a data team(that analyses the data generated by your web app to perform different tasks). Let’s say the web team deploys a feature that replaces a boolean is_customer field with a role field(that has enum CUSTOMER, SHOP_OWNER, ADMIN). The problem here is that during demos at the end of the sprint, the web team would often communicate the “new features” that were added and may forget to talk about the column change.

After deploying such changes, the existing queries of the data team would suddenly stop working or give incorrect results.

Also, imagine that a non-technical teammate(e.g. a marketer) was to ask you when the web team deployed the “google login” feature to the website; where would you find that information?

We experienced these problems in my team at Stears, and I think our solutions to all of them are related. In this article, I would like to share a high-level view of how we solved this.

Below is a summary of the problems we found:

1. Demos usually omits crucial minute changes to the other team members, and we need a way to notify them when we deploy automatically.
2. There was no place to see the history of changes that we made in the past

Here is how the rest of the article structured:

• How we solved this
• Link between changelog and commit convention
• Choosing a commit convention
• Enforcing a commit convention
• Generating a changelog
• Sending the message to slack
• Sending different messages from the pipeline

How we solved this

We deploy new changes to production in our system by creating a release tag. Once a release is created, a manual job in our pipeline, which when run deploys our app to production.

Since our deployment strategy involved creating tags and changelogs contain information about the changes that have been deployed in each release, we decided to solve the problems stated above by maintaining a changelog in our project.
Adding a changelog led us to choose a commit convention, enforce it, and add commands that extract the most recent changes from our changelog and send it to slack whenever we deploy.

Link between changelog and commit convention

Maintaining a CHANGELOG lets us know the releases(and the changes within each release) that have been made historically and allows us to extract the most recent updates easily.

For us, we also wanted our changelog to have:

• a link to the ticket so we can see more details about each change
• a link that shows us the file changes that were made while implementing that feature

Below is an example of how our changelog looks:

I have used dummy messages here.

Notice how the changelog shows the date the web team deployed each release and how it groups the changes based on their type(bug fixes, features, automation, etc.)

Manually updating this information to your changelog is going to be painful.

Since we usually add a description of changes made in our commit messages, it makes sense to use these messages to automate the way we generate our changelog. To do this, however, we needed to have a consistent commit message format(a commit convention) so as to easily extract the data we want to be in our changelog.

Next, let’s talk about how we chose our commit convention.

Choosing a commit convention

When choosing a commit convention, you have to decide what data you want your commit to have. There is a standard called conventional commit which has the following structure

<type>[optional scope]: <description>
<new-line>
[optional body]
<new-line>
[optional footer(s)]

• The “type” section is used to state whether the commit is a feature(feat), refactor, ci change, etc.
• The scope is one or two words that give a general context of the task. This is usually the name of the epic the commit is part of. E.g. authentication, notification, subscription etc
• The “description” is one sentence or phrase that describes the task
• The “body” is where you add more details of the task
• Footers are where you specify extra details like whether the task is a breaking change, ticket number etc.

For us in my team, however, we decided to make slight changes to this by
replacing the ‘footer’ part with a required issue_number and
ensuring that the optional body is in bullet points(when provided).

We did this because we wanted to ensure that the issue number was always in the message we sent to our teammates on deployment.

Below is the structure of the commit convention we chose

<type>[optional scope]: <description>
<required blank-line>
[optional bullet point body]
<required blank-line>
closes docs#<required issue_number>

Examples of valid commits are:

Example 1: scope and body is present

feat(auth): Adds signup page to the app

- adds signup.html file to the app 

closes docs#101

Example 2:: body and scope not provided

chore: Refactors auth folder 

closes docs#121

Example 3:: scope provided but body omitted

ci(deploy): Improve deploy changes

closes docs#121

When we get to the practical section of this series, you will see that you could use any format you want as long as it is consistent and has all the information you want. To ensure consistency, you would have to enforce it.

Enforcing commit convention

It is one thing for everybody to agree on a commit convention, but it is very easy for mistakes and typos to be made when writing commits. Thus, it would be best to enforce the convention you choose on your local machine and on a CI pipeline to ensure that everyone obeys/follows it.

We achieved this by adding a command to our pre-commit hook that validates that our commit messages followed our convention and added the same check to our CI pipeline.

To enforce the agreed commit convention, we used a tool called commitizen which reads a regex specified in a cz.toml configuration file to test each commit message that is created. You will see how we did this when we get to the implementation detail section.

Once we have enforced our commit conventions, the next thing we do is to generate a changelog.

Generating the changelog

The commitizen package also allows you to generate a changelog from your commit messages.

To achieve this, we had to determine how we wanted each bullet point that highlights a change to look, create a grouped regex that extracts the parts of our commit message and use a class-based configuration from the commitizen package.

I will show you how I did this in a subsequent article.

So now we have the valid commits, and we have a changelog its time to talk about slack.

Sending the message to slack

After generating the changelog, the final step is to add code that automatically extracts the most recent release from the changelog and add it to the slack message.

We achieved this by creating a slack bot in our slack workspace and programmatically sending messages to a channel via a slack client.

While doing this, however, we quickly realised that slack does not recognise markdown but has its own rules for achieving rich text formatting via its API. Let’s call this slack technique of formatting messages “slackdown” (a fancy name invented by my manager)

As a result, we created a parser that converts markdown to “slackdown” alongside building a slack client. Once this parser has been completed, sending the slack message becomes easy.

Sending different messages from the pipeline

Once the slack bot has been created and is working, the next step is to develop different commands that would run on different scenarios(or events) in the pipeline. In our case, we needed a CLI command that would send a message on:

• QA-deploy-success
• QA-deploy-failure
• prod-deploy-success
• prod-deploy-failure

Conclusion

Hurray! If you got here, then you are fantastic! This article discussed a lot, and I explained the steps we took to maintain a changelog and notify our teammates when we deploy new features.

In asubsequent article, I will take you through each step with code examples.

You can follow StearsBusiness to get notified once it is created.