Migrating Your Open Source Builds Off Of Travis CI

Starting in early 2021 and continuing to this day, a movement has been underway to migrate open-source projects off of Travis CI. So what happened, and where should you move your project to?

I bet you can't guess where I recommend?

If you're not familiar with Travis CI, it's a build company that has been powering the continuous integration (CI) of many open source projects since it launched in 2011. It was the first build solution that was free for open source use and that easily integrated into GitHub.

What Happened?

In 2019 Travis was acquired by a private equity group and many engineers were let go.

So apparently Travis CI is being strip-mined immediately after their acquisition by Idera. Sorry, I mean after "joining the Idera family" 🙄 https://t.co/CE5ERp1RsY A bunch of talented people are waking up to termination letters. Absolutely shameful. https://t.co/BbBRPdnswe

— Senior Oops Engineer (@reinh) February 21, 2019

Then, on Nov 2, 2020, Travis CI announced the end of its unlimited support for open-source projects:

For those of you who have been building on public repositories (on travis-ci.com, with no paid subscription), we will upgrade you to our trial (free) plan with a 10K credit allotment.

When your credit allotment runs out - we'd love for you to consider which of our plans will meet your needs. - Travis CI blog post

The reason behind the change is stated to be abuse by crypto-miners:

However, in recent months we have encountered significant abuse of the intention of this offering (increased activity of cryptocurrency miners, TOR nodes operators etc.).

However, many feel the real reason is that the acquirer is aiming for profitability at all costs and supporting the open-source community represents a significant cost.

My previous company was on Travis, and as soon as I saw that Travis was purchased by private equity, I knew the downward spiral had begun and I recommended we move to something else. Not surprised that this is happening a couple of years later...my understanding is that private equity will tend towards slowing/stopping development after acquisition to cut costs/headcount, and then squeeze the remaining value from what's left, so this is in-line with that playbook.  - rpdillion on hacker news

Why It Matters

The open source movement runs on the heroic efforts of not enough people doing too much work. They need help. - CLIVE THOMPSON

Many open-source projects are still using Travis and open-source maintainers are notoriously overworked.  Time spent migrating builds is time not spent on other things.  Large well-maintained projects will likely quickly transition but for many smaller projects, an abrupt change in a service they depend on is a huge challenge.

Where to Move To

If you maintain an open-source project that uses TravisCI and are hoping to get off it, then assuming you have the time to migrate, there are actually many viable options.

Option: Run Your Own Builds

You can find some scattered instructions online for running Travis builds yourself. There are mixed reports on the stability and feasibility of this approach, but if your adventurous, you could try to set up your own Travis CI build executor on your own hardware.

A better option, if you want to run the builds on your own hardware is to look at something like Buildkite or GitLab CI.

Option: Circle CI

A better option is Circle CI , a Travis CI competitor which still offers a free plan.  

Circle CI offers 400,000 build credits per month to any open-source public repository.  This is their free plan and limits concurrency to 1 job at a time. They also have an easy GitHub integration and no application process.  

They also allow use of the free plan with private repositories. This makes it a great choice if your project is not actually open-source.

Best Option: GitHub Actions

An even better option is GitHub Actions, a cloud CI system directly from GitHub.  GitHub is at the center of many open source projects and this makes it a natural choice for CI.  

GitHub Actions (GHA) is newer than either TravisCI or Circle CI, having launched in late 2018.

GHA offers very generous build credits, 20 concurrent build jobs per project and no limit on build time used.   If your pipeline can be run in parallel this concurrency can really be a great enabler.  The only limitation I was able to find is that the build may last no longer than 6 hours in total.

If your project is hosted on GitHub and is open source then the GHA open source plan seems like the best bet right now.

But wait, there is an even better solution: being vendor neutral. Before I explain how you become vendor neutral lets review.

Summary of Open Source Plans

Don't Let This Happen Again

So GitHub has a generous build plan, but moving your CI process is not easy or free.  The more complex your build, the harder porting from one cloud CI to another is going to be.  If you move to GHA and then GHA stops being a viable option in the future then this whole effort will have to be repeated.  

Neutral Build Specifications

How can you minimize the effort of moving from build platform to another?

My suggestion is to keep as much logic as possible out of the proprietary build definition. Instead, define it in an open-source format that you can execute anywhere.

Makefiles and Dockerfiles

One way to build a CI neutral build definition is to use a Makefile and a dockerfile.  The Makefile contains the various steps of your build pipeline and you run it inside a docker container which installs any needed dependencies.  QMK is a popular open-source project that uses this approach.

FROM qmkfm/base_container

VOLUME /qmk_firmware
WORKDIR /qmk_firmware
COPY . .

CMD make all:default

QMK Docker File for executing the full build

Earthly

I am an Earthly contributor and this is the Earthly blog, but in my totally biased opinion, it deserves a mention as an neutral format for defining a build. The Elixir web framework Phoenix is a great example to take a look at.

Earthly is like a Makefile where each step is containerized and dependencies are explicitly declared.  

FROM golang:1.13-alpine3.11

build:
 COPY main.go .
 RUN go build main.go
 SAVE ARTIFACT main AS LOCAL main

lint: 
 ...

Example build steps for a go application

Other Interesting Options

Easier Migration from Travis to GHA

Migrating your build out of Travis will take a little work.  If you aren't interested in a neutral format, this GHA action might make it easier.  

This action setups environment variables specified in the .travis.yml file and then runs one of the (potentially) many build jobs within the test build stage.

Serverless Builds

Another interesting option if you are feeling adventurous is using AWS lambda as your build executor.  I have no idea how feasible this is, however, the gg project from Stanford looks interesting.  It attempts to use AWS lambdas for running builds at the maximum possible parallelism.  

Take-Aways

You probably need to move your open-source project's builds off of Travis CI. If you host it on GitHub, GitHub Actions is probably a good choice. There is a risk that the GHA offer will disappear as well.  You can protect yourself from that by defining your build in an open format that is easy to move around.  All build problems can be solved by another layer of abstraction.

If you are going that route, I think Earthly is a great option, but as I said, I am biased.