REMEMBER: gitlab ci runner & pipefail == ouch

#git #bash #pipefail #gitlabci

Any odd failures that make no sense in gitlab-ci.yaml scripts? Maybe those are using pipes? Gotcha.

Gitlab has by default the pretty 'anal' pipefail bash mode fail, which you usually dont see even if you usually use "set -e" ...

So for instance we wanted to truncate certain files in CI to a certain bytesize and use this script pattern:

 - cat /path/to/largefile | head -c 131200 > /path/to/truncfile

Now interestingly enough we always had this command failing our CI scripts if the largefile became substantially larger than the head cut off.

Turned out that cat indeed will fail as its downstream process exits before its done catting the full largefile.

Try it yourself:

$ $ dd if=/dev/random of=/tmp/largefile bs=1M count=3
3+0 records in
3+0 records out
3145728 bytes (3,1 MB, 3,0 MiB) copied, 0,0467213 s, 67,3 MB/s

# here how it works nicely, with return code 0
$ cat /tmp/largefile | head 1024 > /tmp/truncfile
$ echo $?
0

# ... now set pipefail mode it fails ...
$ set -o pipefail
$ cat /tmp/largefile | head -c 1024 > /tmp/smallfile
$ echo $?
141