How to do big upgrades with small changes

Introduction

I'd like to share how I manage multiple library/gem/package update at the same time when I'm working on big upgrades. But before we start, it's important to know why.

Big upgrades require bigger change of files. The amount of files that you need to change can get really out of hand. You may have to replace some code due to the deprecations or maybe you have to deploy in multiple stages in order to not break the production.

Which is why I strive to have smaller changes that can help reduce the final changes. In the end, the final Pull Request will only have small changes that would help your colleagues to review them safely. Yeah, no one is gonna review 50+ changed files.

It's very common for me to have more than 30 PR opened (not at the same time). In fact, it's not that weird to have 10+ PR(s) opened waiting for approval or deployments. So, it can be very confusing to maintain all those PR(s).

It's not just about having small changes, it's also about having the ability to get into the future where you are in a state where you have everything merged so that you can work on the future upgrade at the same time.

In my case, I was working on upgrading v5.2.7 -> v5.2.8 -> v6.1.7 -> v7.0.4 (I'm also upgrading Ruby at the same time, but won't go into that to simplify the explanation). The good thing is that I don't have to wait for everything to be ready before even working on the next upgrade. When I was working on v5.2.8, I have already started to work on v7.0.4 at the same time while waiting for code review and deployments. To me, that's a BIG ADVANTAGE.

How am I doing it

Let's talk about how I'm doing it. Pretty sure someone else has done this before. It doesn't require tools (you can create a script if you want to), just need some basic git commands and some small notes. It's a little bit hard to explain this without a visualization, so, I've created one:

1. Every PR must be based on the master (master <- pkg-upgrade-1 )

This means, you can ensure you won't break production with this particular change and you can always rebase from master whenever needed to. If things went wrong when you deployed, you will immediately know the problem compared to having multiple upgrades in one PR. If you have one PR with multiple upgrades, you'd have to hunt down the part that is causing the problem

1. If the change requires another change, use a different base (master <- pkg-upgrade-1 <- pkg-upgrade-2)

The world is never that simple, there's always a chance where you'd have to depend on a different package before you can work on another upgrade. Obviously you can wait for the first changes to be deployed first, but why should you?

1. Use a 'before' branch before the big upgrade (master <- before-big-upgrade (upgrade1, upgrade2, ..) <- the-big-upgrade)

The big upgrades usually requires multiple changes. We can't use 2nd technique here as that is only suitable for one or two changes. We handle this by creating a branch that will combine every small changes that hasn't been merged to master.

That 'before' branch will be used as the base for the big upgrade PR. This is HOW WE TRAVEL INTO THE FUTURE. You can even see if your changes actually work here. The final branch/PR should pass your CI

1. Use the combination of those techniques to work on another major upgrade

We can combine them and the result would look like this: master <- major-upgrade-1 <- major-upgrade-2 <- major-upgrade-3

It may look simple, but 'major-upgrade-1' PR is a combination of techniques from 1 to 3. A little bit hard to wrap your mind when you read it for the first time, but you guys can take a look at the next image. Hopefully, it will help.

Here are some git command that I use to handle everything that I mentioned:

# Sometimes, I need to update a branch that depends on another branch
# This has to be done before we work on others
# Usually, I squashed and keep one commit
git checkout upgrade-pkg-1; git rebase --onto master HEAD~1
git checkout upgrade-pkg-2; git rebase --onto upgrade-pkg-2 HEAD~1

# Reset to master
# 'before' branch should be refreshed periodically to ensure it won't break production
# It's ok to reset since we are not creating a PR for it
git checkout before-rails7_0_4 ; git reset --hard master


# Merge those branch that hasn't been approved / wip
# Normally, I have close to 10 package that needs to merged
git merge upgrade-pkg-2 \ # I'm skipping pkg-1 because this pkg-2 has the changes
upgrade-pkg-3 \
upgrade-pkg-4

# Sometimes, not all packages can just be merged
# This requires manual merge due to the conflicts
git merge pkg-5 # resolve conflict

# The straightforward verion
# Rebase final branch to the 'before' branch
git checkout am-upgrade-rails7_0_4; git rebase --onto am-before-rails7_0_4 HEAD~1

# This can happen:
# I had to do something like this when the upgrade has major conflict
git checkout am-upgrade-rails7_0_4 ; git reset --soft HEAD~1
git reset
git checkout Gemfile.lock
git add .; git stash; git reset --hard am-before-rails7_0_4
git stash pop # fix conflicts
bundle update rails rails-i18n
git add .
git commit -m 'Upgrade Rails from v6.1.7 to v7.0.4'

Is this the best way to do this? I'm not sure, but it certainly helped me to keep on working without being blocked by pending PR or deployments. Do remember, I don't deploy immediately to ensure I can monitor for regressions. Usually, I'd put some gaps between deployments.

I highly doubt people will read this far lol, but if you did, thanks! I'm planning to do a presentation on this in the future and writing this will certainly help me to explain the method better.