Turns out that a very old test was broken for a while and wasn't running in the automatic build, whoops.
It would be very helpful to find the exact commit that broke the test. Well, git-bisect to the rescue: git bisect will binary search the commit history to find the bad commit. And we can use pytest to tell git if the test is passing or not for any possible commit.
Set Up
Here is a toy repository with a simple function and a unit test:
$ ls
my_script.py README
And this is my_script.py:
from functools import reduce
from operator import mul
def product(factors):
return reduce(lambda x , y: x - y, factors)
def test_product():
assert product([1, 2, 3,]) == 6
Pretty simple so far. test_product broke for some reason.
Let's look at the commit history:
$ git log --oneline
7e26872 Unrelated commit # 21
f67f19e Unrelated commit # 20
...
2b37410 Unrelated commit # 15
80956a1 Unrelated commit # 14
d2a3327 Some innocent change that certainly did not break anything
8cc3f8b Unrelated commit # 13
1b09ced Unrelated commit # 12
...
3e73dfa Unrelated commit # 2
790ba87 Unrelated commit # 1
0c75c27 unrelated commit
d7af196 Added product function with a unit-test
Bisecting
There are several ways to use bisect. We are going to tell it when the test was passing, and how to run the test, and it will take care of the rest.
We begin with bisect start:
$ bisect_start
Tell it that our current commit is broken:
$ git bisect bad HEAD
Let's find a commit where we know the test was passing. Say, the commit that introduced the test:
$ git blame my_script.py | grep test_product
d7af196a (oren 2019-06-24 12:48:35 -0700 8) def test_product():
Tell bisect this commit is good:
$ git bisect good d7af196a
Bisecting: 11 revisions left to test after this (roughly 4 steps)
[97c3a24f28f72bec1885445eb3947e446d0804fe] Unrelated commit # 10
Now we start bisect off with a command to run the test, bisect uses the exit code of the command to tell whether each commit is good or bad, that works because pytest has a non-zero exit code if tests fail (And, pytest can run a specific function):
git bisect run pytest -qqq my_script.py::test_product
....
....
....
d2a3327df55cb9bf9b43780f9adb223e93ba093d is the first bad commit
commit d2a3327df55cb9bf9b43780f9adb223e93ba093d
Author: oren <----@----.--->
Date: Mon Jun 24 12:50:58 2019 -0700
Some innocent change that certainly did not brake anything
:100644 100644 0be2cd4c2d0e450026507ee86e058785de56cadc 98c1b14d74c4f46b538df5cc1d32b9bb42afc791 M my_script.py
bisect run success
Cool!
And indeed, this is the bad commit:
$ git show d2a3327df55cb9bf9b43780f9adb223e93ba093d
commit d2a3327df55cb9bf9b43780f9adb223e93ba093d
Author: oren <----@----.--->
Date: Mon Jun 24 12:50:58 2019 -0700
Some innocent change that certainly did not brake anything
.......
.......
def product(factors):
- return reduce(mul, factors)
+ return reduce(lambda x , y: x - y, factors)
Note: if you are using a script as a command to git bisect run. Make sure this script is not tracked by git, otherwise it might change when git changes revisions.
Cheers!
Top comments (2)
Hello! What is the bisect_start program/command?
the
bisect startcommand puts git in thebisectstate. It will then expect you to tell it what are the good and bad commits. Git won't actually run anything afterbisect start.It is only after
bisect runthat git starts searching and running the command you provided it on different commits.It is also possible to provide the good and bad commits as parameters to the bisect start command.