Recently I wrote a binary called "Watchdog" in Rust on my Mac that I wanted to take with me to Linux systems I frequent, but I couldn't bring a Rust compiler to these systems for unrelated reasons. This meant I had to cross-compile my application and ship just the binary.

Now, I'm taking a few liberties in this post to showcase this as a proof-of-concept, but most notably of these is that I am not handling dynamically-linked libraries. I'm shipping a standalone binary, that is to say the one file I build has everything it needs to run, assuming you run it on the correct platform. Or, so I naively hope - it can definitely be more complicated than that.

What is cross-compilation?

Cross-compilation is compiling your code on one platform (for me, Mac OS) to run on another platform (for me, Linux). If you're more familiar with languages like Ruby, JavaScript, and Python, this might not make a lot of sense since your code already runs on different platforms most likely. But for compiled languages like Rust, C++, C, etc., we have to compile it to code that can run on certain platforms. That's what we're doing here, we're compiling my Rust code on my Mac to run on another computer that runs Linux.

You can configure cross-compilation on your local computer, but it can be difficult, and often has some big quirks. I've dealt with this recently when cross-compiling a lambda function in rust for mac->linux. In particular, getting OpenSSL to behave was an absolute nightmare and killed my motivation for months. Eventually, I kicked the problem to the curb and said I'll bring in Docker just to work around it. Your mileage may vary.

So instead of xcompiling locally, we'll leverage Travis CI to do this natively for free. Technically it's not cross-compiling, since Travis will compile it in a Linux VM, but the accomplished goal is the same - building my application for platforms I can't otherwise. So, if you'd like to chew me out for click-baiting, I welcome it - I just couldn't think of a better way to word what we're doing here!

Where Is Your Code?

You can see all the code I worked on for this post on my GitHub project!

USAGE:
    watchdog [FLAGS] [OPTIONS] <COMMAND> [-- <PATH>...]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose    Enables verbose output

OPTIONS:
    -g, --glob <GLOB>    Glob used for matching files

ARGS:
    <COMMAND>    Command ran on response to changes
    <PATH>...    File path(s) used for matching files

Initial Travis Config

Travis lets us configure it via code, so my initial .travis.yml file before I set up "cross-compilation" looked like this:

language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
deploy:
  # This part's not very relevant to this blog post, we'll add a new deploy step though, so I'll keep it as an example of what you might already have
  provider: cargo
  skip_cleanup: true
  on:
    tags: true
    condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
    branch: master

If you're not familiar with Travis, I'll give you a brief summary of what this config does.

1. We run a "matrix" build (multiple platforms and rust versions) to test both stable rust versions and nightly builds of their rust compiler on different platforms.
2. Nightly builds can fail, since those compilers aren't final and may have bugs, so don't mark these build fails as an overall failure if that's all that went wrong
3. When I tag a release in git, publish this as a new version of the crate (rust package) in the global repository so other rust users can grab the latest version just released

Note that there's no talk of publishing binaries yet - because that's the new part we'll add today!

Travis Releases Provider

GitHub already has a sweet releases feature, and I happen to be using it for Watchdog, so I'd like to publish my built binaries there so users can find it easily if they, like me on my linux machines, can't access a Rust compiler.

Travis CI is super rad, so naturally they already have swell support for GitHub Releases! Lucky us!

To use this provider, we'll have to:

1. Create a new personal access token on GitHub for Travis to use to upload the binaries to your releases: https://github.com/settings/tokens
2. Add them to your Travis project:
   • In your terminal, run these commands:
   • cd <Your project>
   • gem install travis (you'll of course need ruby + rubygems, which Mac OS comes with)
   • travis encrypt (then follow the provided instructions)
3. Add the below code to your .travis.yml:

deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file: target/release/watchdog # where `watchdog` is your built rust binary
    api_token:
      secure: # this part from your `travis encrypt` command

Testing Travis Changes

I'm no Travis expert, so maybe a commenter can help us both learn a better way to test this, but here's how I test changes to my travis config:

1. Commit the change in git
   • To save your sanity, comment out the branch: master part so you can do this on a throw-away branch
2. Tag the commit
3. Push to GitHub (remembering to push with tags, i.e. git push --tags!)

Travis will then run your new build.

Connecting the Dots

The above config won't quite work for us though, so we have to be a little more clever about what we send to GitHub.

If you added that last section like I suggested, you'll notice that target/release/watchdog doesn't exist. This is because by default, Travis builds in Debug mode (notice the absence of Rust's required --release field for release mode builds in the "Default script" setting, implying it's Debug):

We want to publish our binary in Release mode, so let's modify our yaml to build in Release mode by adding this key to the root of the yaml:

script: cargo build --verbose --release; cargo test --verbose

Sweet. Now if you tested it, you should see that Travis is a little slower (release builds take a bit more work!) but your file should be uploaded to the GitHub release! 🎉

But Wait, There's More!

Notice how we build for two platforms, but only see one binary in the release? And we can't tell what platform its built for because its just named watchdog? That's no good, if I downloaded that on my linux box, I have no idea if I'm downloading a linux binary or a mac binary.

Why is there only one?

Well, admitting that I didn't encounter this myself (so your exact error may vary) because I did this out of order, these Travis builds run out of order and concurrently, so your linux build may upload a file named "watchdog", only to be overwritten by your mac build when it uploads a different file named "watchdog". So, we have a race condition.

How can we get both binaries to upload and not overwrite each other?

Well, we can get two birds with one stone here, by also indicating in the file name whether it is built for linux or mac. The idea is we have to give the files on different builds different names so we don't overwrite them, so we might as well specify which platform they're built for to be clearer to users too.

Renaming Your Binary

This was the fun part for me, tracking down how to rename binaries so we could upload multiple to GitHub and specify their platform. It only reinforced my love for Travis when I learned that it provided all the building blocks I needed:

• a before_deploy script step to run whatever I want after we run the script step yet before we run the deploy step
• a collection of environment variables to tell all sorts of things about our builder, like its platform

With that in mind, we can just connect the dots to rename the binary before we upload it in the deploy step:

before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"

But, if we want to support having multiple deploy steps (publishing to crates.io AND GitHub releases, for instance), this will actually cause an error and fail the whole build. That's because this before_deploy script gets called once for each deploy step. That actually burned me and caused my "final" build to fail, because the second time it was called, mv threw an error because target/release/watchdog didn't exist anymore. So, to account for that, you'll need a little bash-fu to update your yaml once more:

before_deploy:
 - "if [[ -f target/release/watchdog ]]; then mv target/release/watchdog \"target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME\"; fi"

What this modification does is use bash to only run mv if the file exists, so we won't error out and fail the whole Travis build if we have multiple deploy steps. Anyways....

Rad, now our binary is named something like watchdog-v0.2.0-linux! Now we can adjust our deploy step to find this new name pattern:

deploy:
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file_glob: true # <-- note this new field
    file: target/release/watchdog-*  # <-- note the `-*`
    api_token:
      secure: # this part from your `travis encrypt` command

The key here is to use globbing to find the new binary, since we can't hardcode the name anymore. We have to enable that with file_glob: true, then we can add -* to the end of our file name to find the new name pattern easily!

Testing this now should show a GitHub Release with your two platform's binaries both uploaded, finally. Way to go! 🎉

Travis	GitHub

Putting it All Together

Here's the final Travis config we used:

language: rust
rust:
- stable
- beta
- nightly
matrix:
  allow_failures:
  - rust: nightly
  fast_finish: true
os:
- linux
- osx
before_deploy:
  - mv target/release/watchdog "target/release/watchdog-$TRAVIS_TAG-$TRAVIS_OS_NAME"
deploy:
  - provider: cargo
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable && $TRAVIS_OS_NAME = linux"
      branch: master
    token:
      secure: # ;)
  - provider: releases
    skip_cleanup: true
    on:
      tags: true
      condition: "$TRAVIS_RUST_VERSION = stable"
      branch: master
    file:
      - target/release/watchdog-*
    api_key:
      secure: # ;)

And that's it! Now we've got a Travis config that allows us to build and publish binaries for any platform Travis supports whenever we tag a release.

You can check out my release I cut to demonstrate this here: https://github.com/hawkins/watchdog/releases/tag/v0.2.5

Once more, you can review all this in action with a real Rust project on my GitHub. And if you have any questions, feel free to leave either a comment on this blog post, and issue on the below repo, or shoot me an email!

USAGE:
    watchdog [FLAGS] [OPTIONS] <COMMAND> [-- <PATH>...]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -v, --verbose    Enables verbose output

OPTIONS:
    -g, --glob <GLOB>    Glob used for matching files

ARGS:
    <COMMAND>    Command ran on response to changes
    <PATH>...    File path(s) used for matching files