Git Bisect is Easy (How to Initiate the Robot Uprising)

When we find a new bug in our applications or a test randomly fails in CI, frequently, the bug came to exist as the unintended side effect of another change.

There are a lot of blog posts and documents out there about avoiding these unintended consequences, but since none of us are perfect it's important to know how to diagnose and locate the root cause of this category of bug.

The good news is, if you're using git, it's easy. If you're not using git, stop reading this and go run git init in your project's directory.

Because developers frequently need to find the change that introduced an undesired behavior, there is a git command called bisect. Bisect means, "to cut or divide into two equal or nearly equal parts." That's pretty much what this command does.

Bisect means, "to cut or divide into two equal or nearly equal parts."

Before we get into git bisect, let's set up a basic project to illustrate this problem. I'm going to do this with Ruby, but it's not super important what language the example is in.

If at any point, you are confused by the code in this article, just skip it. I'm just using it to explain why you might need to use git bisect.

I'm going to make a directory and run git init, then I'll write a function and test for that function. That's our starting point.

mkdir bisect-example && $_
git init
vim main.rb

# main.rb

def square(number)
  number * number
end

# Testing Code

def assert_equal(actual, expected)
  unless actual == expected
    raise "Assertion failed: #{actual} does not match expected #{expected}"
  end
end

def test_square
  assert_equal(square(2), 4)
end

test_square

So, we've got a function called square that has one parameter and it does what you think it does (multiplies the argument passed to it by itself). We've got a couple of different functions testing the square function.

Everything is good, so we make a commit.

git add .
git commit -m "Add square function"

The world is in perfect balance until we realize that math has changed, so our stakeholders come to us and say, "With this new math when you square something, you aren't allowed to multiply it by itself, you need to use addition instead."

So we refactor to our function.

# main.rb

def square(number)
  result = 0
  number.times do
    result += number
  end
  result
end

...

Now we have this gross implementation, but our stakeholders are happy and tests are passing. So we commit it.

git commit -am "Refactor square function to meet requirements"

This happens a dozen more times (and our coworker who doesn't see the value in good commit messages does some of the work), then some new features are added to help us get extra VC. Now the project is really hard to wrap our heads around, but because we are smart developers we wrote tests! We can rely on those to help us identify when there is a problem.

One day, we are happily working in our main.rb file and we run tests to find that somehow our square function is no longer functioning the way our tests expect it to function! Oh no!

main.rb:15:in `assert_equal': Assertion failed: 5 does not match expected 4 (RuntimeError)

Our current changes aren't even touching the square function, we are working on a whole new function called overthrow_humanity which doesn't even use square. Because we are smart developers, we decide to look at the git history for insight into how this could have happened:

git log --oneline --no-merges

f25411d9e9 Add README
9a8225f6b3 Remove unused code in RobotOverlord#simulate_mercy
708a4955e0 workaround for legacy T-800 model
9a2f2805f6 this probably works
360689a0ec Add missing tests for AI features
cfcb762e91 fix whitespace
32ca0fa720 implement features because PO said to
26dd1ab393 Fix bug in cube function
11e118d981 Fix bug in square function that causes hostility
01da7be780 Add test coverage for RobotOverlord#enslave
59415871b2 Add dependency on skynet
31bb9ac6c2 do some work
92f4123391 Add RobotOverlord class
a1e8a90763 Improve test coverage for math functions
069ab50172 Refactor square_root? function
18512a7fef Add square_root? function
a15700afb3 Add cube function
c25f8ef431 Refactor square function to meet requirements
12e4ebffce Add square function

The last four or five commits don't seem to be very helpful in diagnosing the issue, in fact, at a glance none of them even touched the main.rb file. That doesn't mean those changes couldn't have unintended side effects, so we need to identify which change actually broke the test.

We have finally arrived at git bisect.

The first step in fixing the code that broke this test (we are going to assume the test is right), is identifying how exactly it got broken. In one of these commits, a piece of code was added to the project that broke this test. How do we determine which commit housed that code? Easy, run the tests on each commit until we find the one! What if it was 200 commits? Not quite as easy.

The command git bisect takes advantage of binary search to quickly find a commit that introduced a given bug. Let's test it out on this project.

git bisect start

We've begun our bisect, but we need to tell git that the current state of the project isn't working as intended. To do this, we mark the current commit as bad.

git bisect bad

The next piece of information git needs is a commit where we know our code was working as it was intended. Well, we remember running the tests and being satisfied the first time we had to refactor the square function, so let's go with the commit where did that: c25f8ef431 Refactor square function to meet requirements. We need to tell git, that c25f8ef431 was good.

git bisect good c25f8ef431

You should see a response that resembles this:

Bisecting: X revisions left to test after this (roughly Y steps)
[some_commit_hash] Some commit message

Now it's as simple as walking through each revision, running our tests, and marking those commits as good or bad.

When you've run your tests and identified that the commit you're on is good, just type: git bisect good. When you find a commit that is bad, type: git bisect bad. The software is going to do the hard work of finding the commit that actually broke our test.

In our example project, we are going walk through several revisions and we will find that in one of our commits (11e118d981 Fix bug in square function that causes hostility) someone messed with the square function.

Bisecting: 0 revisions left to test after this (roughly 0 steps)
[11e118d981] Fix bug in square function that causes hostility

When we look a little deeper to see what changed in this commit, we will see that some developer (probably us) tried to fix a bug elsewhere in the project by modifying our square function.

# main.rb

def square(number)
  result = 1          # 🚨 This is the bug!
  number.times do
    result += number
  end
  result
end

...

Now we know exactly what caused our bug. We can ask git to take us back to where we started the bisect by typing:

git bisect reset

Now we just have to fix the bug and commit the change.

That's how you use git bisect to take advantage of binary search and find a commit that introduced a bug! This is a contrived example, but it really is that easy.

It's on you to write tests that allow you take advantage of git bisect in this way.