TLDR: Randomize the execution order of tests and benchmarks with -shuffle off,on,N
flag.
As a general guideline, it's bad for unit tests to depend on each other and the order of them messing with the results.
A "trick" I have seen some codebases adopt to mitigate and catch issues with unit test results relying on order is instead of using arrays in table-driven tests using maps.
TestMyFunc(t *testing.T) {
for name, _ := range map[string]struct{}{} {
t.Run(name, func(t *testing.T) {
...
})
}
}
The reason behind this is, the order of map iteration is not guaranteed, so on different runs the tests might run in different order making it more difficult to rely on the order.
But as of Go 1.17, you can "Randomize the execution order of tests and benchmarks."
As go help testflag
states
-shuffle off,on,N
Randomize the execution order of tests and benchmarks.
It is off by default. If -shuffle is set to on, then it will seed
the randomizer using the system clock. If -shuffle is set to an
integer N, then N will be used as the seed value. In both cases,
the seed will be reported for reproducibility.
Top comments (0)