Working with Capistrano: Environment Variables and Remote Commands

In the "Working with Capistrano: Tasks, Roles, and Variables" article, we explored Capistrano's fundamental concepts. We wrote two useful tasks. One for logging into a server and another for accessing Rails console in a production system. This time around, we're going to write a more involved script composed of many tasks. By themselves, these tasks won't accomplish too much. But the sum of them all will help us automate the installation and upgrading of Ruby throughout the lifetime of our application.

Not so long ago, every deployment server that I worked with had only one version of Ruby installed. This system-wide installation forced all Rails applications to use a very specific version of Ruby, for example, 2.1.5. You were not only stuck with a Ruby version but also married to a particular Rails release that had a minimum Ruby requirement. It doesn't take much to realise the shortcomings of this approach. Any Ruby upgrade requires a new production server setup and a costly migration effort. Who has the time and money for this? From a reliability point of view, Ruby becomes a single point of failure. If Ruby were to become corrupted in some way then this would affect all the web applications on the same server.

The way to ease this pain is to use a Ruby version manager. A tool that will allow many different versions of Ruby coexist on a single server. What follows, each Rails application can have its own Ruby environment and be updated independently. Developers can easily change Ruby during the lifetime of their application without affecting other projects - a nice bonus.

There are many great tools such as RVM or chruby that can help you manage Rubies. But, rbenv is my preferred choice. It strikes a perfect balance between features and simplicity. The project is also maintained by Ruby core team members and that's what I will demonstrate in this article. While we explore rbenv, we will lift the veil and examine more Capistrano techniques to add to your toolbox.

In the code samples that follow, we'll install files on a server in some places and read them back from others. I'll assume a Unix like operating system such as Linux since that's what I'm most familiar with. You may need to adjust a few bits according to your operating system of choice.

With that out of the way, let's dive into rbenv installation.

Where to Install rbenv?

Before we decide where to install rbenv, we will want to name and organise our new task first. A great feature that Capistrano inherits from Rake build tool is the namespace method. It gives us a way to logically group tasks under one name. A side benefit is that we can reuse similar task names that otherwise could conflict with the built-in ones. I tend to group the tasks based on the overall function that ties them together. The rbenv as a namespace seems to fit well. We'll go ahead and inside lib/capistrano/tasks directory create a file called rbenv.rake with the :rbenv namespace:

# lib/capistrano/tasks/rbenv.rake

namespace :rbenv do
  ...
end

To install rbenv, we need to decide where we want it installed first. There are two primary ways to install rbenv: system-wide and per user account. What's the difference? The system-wide installation means that a given Ruby version is installed only once per server in a central location. For example, on a Unix system that would be /usr/local/rbenv. Conversely, user account installation restricts a given Ruby to locations accessible to that particular user. Usually, this is a hidden folder inside the user's home directory, for example, $HOME/.rbenv. Choosing the latter may lead to having similar Ruby versions exist on the same server. It is a balancing act.

As we discussed before, the system-wide installation suffers from a single point of failure issue. Because of this, by default, we're going to install rbenv for each deployment user account. But we will expose :rbenv_path variable to allow configuration of a custom path. To do so, we will create a helper method rbenv_path that will hold a rbenv installation path, by default, set to the user's home directory:

def rbenv_path
  fetch(:rbenv_path, "$HOME/.rbenv")
end

The fetch method accepts as a second argument, a default value that will be used when no value is set in your script. There is another way we could do this by using a block.

def rbenv_path
  fetch(:rbenv_path) { "$HOME/.rbenv" }
end

The difference is that the default value provided via the block will be lazy evaluated. This means that the evaluation of default value will only happen when the rbenv_path method is first used. This comes handy in a situation when you have, for example, a dynamic value that depends on the evaluation of other methods. We are going to stick with the second argument option here.

One aside: when working with the filesystem paths, it's better to represent them as data types rather than strings. What does this mean? A string is a poor representation for a system path. It doesn't give you any extra information, for example, whether a path is relative or absolute. Ruby's standard library includes Pathname that can wrap path string representation. Unlike string, the pathname object can tell us information about the path it represents and enhance our path operations. We can build paths in a system-independent way. Joining two paths is as easy as adding them together. The pathname object will automatically use the correct file path separator. If you want to learn more, I recommend Paths aren't strings article.

Armed with this knowledge, we change rbenv_path to return a pathname object instead:

# lib/capistrano/tasks/rbenv.rake

require "pathname"

namespace :rbenv do
  def rbenv_path
    Pathname.new fetch(:rbenv_path, "$HOME/.rbenv")
  end
end

Now, we need to find a way to tell Capistrano to use the rbenv_path method to find rbenv executable and make it available for other tasks to use.

The PATH Variable

When deploying applications, Capistrano uses what is called a non-login shell. In other words, Capistrano doesn't assume anything about the underlying system apart from establishing an SSH session. There is no visual terminal attached to this session. None of the shell configuration files, like a Bash profile file, are loaded. This is very much desirable behaviour that makes our deployments safe and repeatable. But it also means that we cannot rely on any configuration environment variables to be preloaded and available for our scripts.

Specifically, we cannot rely on the system to tell our scripts the location of rbenv binary installation. How can we work around this? We need to roll up our sleeves and show Capistrano where rbenv is installed ourselves. The way we can do this is by modifying the $PATH environment variable. This variable holds a list of directories delimited by a colon that the Unix shell uses to search for executable files when running commands. To check the value of your path, you can run the following in your shell:

$ echo $PATH

Your path may look something like this:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

This begs the question...how can we instruct Capistrano to change what locations does the $PATH variable point to? Remember, we're trying to make rbenv available for the non-login shell.

Mapping Binaries with SSHKit

The heart of Capistrano - the sshkit Ruby gem - is a toolkit for working with servers via SSH. The sshkit is responsible for doing the heavy lifting in terms of defining and executing commands on a server. Besides, it provides access to the current SSH session configuration with the SSHKit.config method. In particular, the config exposes the default_env method that, by default, returns an empty hash. When a command gets executed, all the keys and their values are converted into the corresponding environment variables. Bear in mind that only the keys provided as symbols will change to an uppercase name:

SSHKit.config.default_env[:path] = "$HOME/.rbenv/bin:$PATH"

This is equal to:

SSHKit.config.default_env["PATH"] = "$HOME/.rbenv/bin:$PATH"

Both examples will result in a command being prefixed with a path:

$ PATH="$HOME/.rbenv/bin:$PATH"

The default_env is exactly the place to change the $PATH variable to include the location of rbenv installation path. Specifically, we want to add two directories with executables:

• $HOME/.rbenv/bin
• $HOME/.rbenv/shims

Now, we're ready to get our hands dirty and make Capistrano locate rbenv. We begin by adding a task called map_bins that will make common Ruby binaries use rbenv executable - but we're getting ahead of ourselves. Let's change the $PATH variable first.

# lib/capistrano/tasks/rbenv.rake

require "pathname"

namespace :rbenv do
  desc "Map Ruby binaries to use rbenv"
  task :map_bins do
    ...
  end

  def rbenv_path
    Pathname.new fetch(:rbenv_path, "$HOME/.rbenv")
  end
end

With the task defined, we read the current value for the :path key to preserve what other tasks may have already configured:

desc "Map Ruby binaries to use rbenv"
task :map_bins do
  path = SSHKit.config.default_env[:path]
end

You may have noticed that we haven't used the on scope. The scope is only needed when we want to execute remote commands on a server or access the server's configuration. None of this is necessary for the task at hand.

Next, we alter the path with the location of rbenv directories. We use rbenv_path helper to add our two rbenv locations. If the path isn't configured, we prepend the two directories directly to the $PATH variable. If the path is configured, we prepend on top of it:

desc "Map Ruby binaries to use rbenv"
task :map_bins do
  path = SSHKit.config.default_env[:path]
  path_env = "#{rbenv_path.join('shims'}:#{rbenv_path.join('bin')}:" +
             (path.nil? ? "$PATH" : path)
end

Finally, we override the SSHKit default_env hash with a new modified environment path:

desc "Map Ruby binaries to use rbenv"
task :map_bins do
  path = SSHKit.config.default_env[:path]
  path_env = "#{rbenv_path.join('shims'}:#{rbenv_path.join('bin')}:" +
             (path.nil? ? "$PATH" : path)
  SSHKit.config.default_env.merge!(path: path_env)
end

But, we're not done yet. If we're going to do anything useful a server like installing Rails, we need to have a way to run Ruby commands like gem or bundle. This is necessary so that we can later issue the following commands to bootstrap a Rails application:

$ gem install bundler
$ bundle install

Let's say we want to run "bundle install" command but there are many Ruby versions installed already. Which version of Ruby would provide the bundle executable? We need to somehow tell our shell which Ruby version to use. Again the $PATH variable comes to the rescue. We can extend the $PATH with the correct Ruby version before running our executable. For example, to use bundle executable from the Ruby 2.6.3 installation, we would run the following in the Unix shell:

$ PATH="$HOME/.rbenv/versions/2.6.3/bin:$PATH" bundle install

But, this command is long, ugly, and hard to automate. Thankfully, the rbenv provides rbenv exec command that can detect the currently loaded Ruby version and include it in our path for us. Since we have already configured Capistrano to find rbenv binary, we can use exec to rewrite our previous command like this:

$ rbenv exec bundle install

Now we know how to write our commands by hand. But how do we tell Capistrano to prepend common Ruby executables like bundle with the rbenv exec command for us? Before we answer this question, it's helpful to understand how an execute method works first.

Running Remote Commands

The sshkit provides an execute method for running any shell command on a server. This is a powerful method that behaves differently depending on the arguments passed in. A peek under the covers reveals a rather succinct and short implementation:

# lib/sshkit/backends/abstract.rb

def execute(*args)
  options = args.extract_options!
  create_command_and_execute(args, options).success?
end

The above snippet doesn't look like much but there is a whole machinery hidden behind the execute method. Conceptually, the process of running any command is split into two parts. The first part takes the command and converts it into an SSHKit::Command object. This object handles assembling a full, shell-ready command from various bits like prefixes, flags and arguments. The second part takes care of executing the command which can be carried away by any of the SSHKit::Backends. This approach lets you create command once and run it using different backends on many servers. This is such a great design decision!

When run with a 'simple' string without any whitespace, the execute hands over the command first to SSHKit::CommandMap. Internally, the CommandMap holds a hash of commands, empty by default, that allow additional configuration:

# lib/sshkit/command_map.rb

module SSHKit
  class CommandMap
    def initialize(value = nil)
      @map = CommandHash.new(value || defaults)
    end
  end
end

By default, if there are no custom prefixes specified, a command is prefixed with "/usr/bin/env". Except for if, test, time and exec, commands that receive no prefixes. For example, if we were to check Ruby version using ruby executable as the first argument:

execute :ruby, "--version"

This would result in the default lookup and the following output:

/usr/bin/env ruby --version

If, on the other hand, we were to provide a command as a raw string with whitespace:

execute "ruby --version"

It would be executed as-is without any prefixing, shell escaping or directory changes. Basically no involvement from Capistrano or SSHKit whatsoever:

ruby --version

This often tricks newcomers to Capistrano who scratch their head in puzzlement as to why a command fails to run on a server. So consider yourself warned.

Prefixing Commands

To change commands before they are run, we can use the command_map method exposed by the SSHKit configuration object. We can then take the CommandMap and chain it together with the prefix method to add new prefixes. Like CommandMap, the returned PrefixProvider stores a hash of command to prefix mappings. By default, any key lookup for non-existing command returns an empty array. Knowing this, we can override a default ruby command prefix:

SSHKit.config.command_map.prefix[:ruby] << "rbenv exec"

Then run our ruby command again:

execute :ruby, "--version"

But this time, we get a very different result:

rbenv exec ruby --version

That was a long but necessary diversion from the map_bins task implementation. Now, we know how to prefix common executables like bundle, gem, or rake with rbenv exec. First, we will add rbenv_map_bins method to allow for the configuration of Ruby binaries defaulting them to the most common:

def rbenv_map_bins
  fetch(:rbenv_map_bins, %w[bundle gem rails ruby rake])
end

Armed with our knowledge of customising commands, we can prefix each binary and ensure we don't change any existing prefixes:

rbenv_map_bins.each do |exe|
  SSHKit.config.command_map.prefix[exe.to_sym].unshift("rbenv exec")
end

Having prefixed Ruby binaries, the complete task for mapping commands will look like this:

desc "Map Ruby binaries to use rbenv"
task :map_bins do
  path = SSHKit.config.default_env[:path]
  path_env = "#{rbenv_path.join('shims')}:#{rbenv_path.join('bin')}:" +
             (path.nil? ? "$PATH" : path)
  SSHKit.config.default_env.merge!(path: path_env)

  rbenv_map_bins.each do |exe|
    SSHKit.config.command_map.prefix[exe.to_sym].unshift("rbenv exec")
  end
end

Now running "bundle install" (and similar) commands will work because they will be executed with the correct Ruby version. We can apply our task by running:

$ cap [stage] rbenv:map_bins

What we have configured so far is for the non-login shell. But, if a user wants to log into a server, the rbenv executable won't be available and none of the Ruby binaries will work. Let's change this!

Modifying Shell Startup File

It would be annoying to force anyone who logs into a server to manually run rbenv setup instructions before they could run, for example, a bundle command.

So what's the solution?

When a user logs into an operating system, this triggers loading of the shell startup files. A shell startup file allows for execution of arbitrary shell scripts, setting environment variables and so on. Sounds like just what we need. We could modify the shell startup file to set rbenv paths for us. This would make Ruby and any related executables available inside the logged-in user shell.

Many Linux distributions like Debian, typically, use a Bash shell as their default shell. When started as an interactive shell, Bash reads a startup file called bash_profile. The following Bash script is all we need to insert into a bash startup file to load rbenv automatically:

if [ -d ~/.rbenv ]; then
  export PATH="$HOME/.rbenv/bin:$PATH"
  eval "$(rbenv init -)"
fi

This snippet of code first checks that rbenv directory exists. When it does, it adds rbenv binary to the $PATH variable and makes it available in the shell. It then runs the rbenv initialisation script.

Now, we have everything we need to create a task called modify_shell_file that will, as you've probably guessed, modify a shell startup file. We use the on scope with all role to specify the servers affected by our changes:

# lib/capistrano/tasks/rbenv.rake

namespace :rbenv do
  desc "Map Ruby binaries to use rbenv"
  task :map_bins do
    ...
  end

  desc "Change shell startup file to setup rbenv"
  task :modify_shell_file do
    on release_roles(:all) do
    end
  end
end

Testing Remote Commands

What we don't want to do is to add rbenv initialisation script to a startup file more than once. If the script is already present, we want to skip adding it. We could use the execute method to run this check but that wouldn't get us far. Why is that? The execute method, though powerful, is used for running commands where we don't care about capturing the final output. We only care that a command succeeds and the script continues or it fails and terminates a task. But, in a situation when we need to decide if a command ran successfully or failed without stopping a task, Capistrano provides the test method. This method is equal to execute with the difference that it doesn't raise an error for non zero exit code. There is even :raise_on_non_zero_exit option you can use to get this behaviour with execute method:

execute ..., raise_on_non_zero_exit: false

To scan a file for matching content, we will use the grep command. It is a basic Unix utility that should already be installed on a server. If a match is found, the grep will finish with a zero exit code - exactly what we need. To skip the task and prevent further execution, we call next. If this task is part of many, the script will continue with the next task. This is a common idiom used in Capistrano plugins. It's worth noting that break or exit would immediately terminate the entire script and raise an error.

So the check to see if a startup file contains rbenv initialisation script looks like this:

desc "Change shell startup file to setup rbenv"
task :modify_shell_file do
  on release_roles(:all) do
    next if test "grep -qs 'rbenv init' ~/.bash_profile"
  end
end

We need to jump through some Unix shell hoops to programmatically insert a multiline string. When executing a multiline command, Capistrano replaces every newline character with a semicolon to essentially collapse a command into a single line. To prevent this and insert rbenv script line by line into the bash startup file, we're going to use the printf utility. If you used Ruby's printf method, the Unix printf utility is going to be very familiar. They both use a format string followed by input arguments, the difference being shell uses whitespace to split inputs.

Let's append the rbenv initialisation script to the Bash profile file:

desc "Change shell startup file to setup rbenv"
task :modify_shell_file do
  on release_roles(:all) do
    next if test "grep -qs 'rbenv init' ~/.bash_profile"

    execute %q{printf "%s\n  %s\n  %s\n%s" 'if [ -d ~/.rbenv ]; then' 'export PATH="$HOME/.rbenv/bin:$PATH"' 'eval "$(rbenv init -)"' 'fi' >> ~/.bash_profile}
  end
end

Having configured rbenv for the login and non-login shell alike, we're ready to install rbenv itself. It was quite a bit of a setup but I promise it was worth it.

Installing Rbenv & Rbenv Build

At this point, we're not going to reinvent the wheel and instead use a rbenv installer. The rbenv installer includes a doctor script that will verify that rbenv is correctly configured. If it isn't then the installation script won't proceed any further and return non-zero exit code. This will stop the installation process in its tracks.

The benefit of using rbenv installer is that it will also install the rbenv-build tool for compiling and installing Ruby. Rbenv by default doesn't include any scripts for installing Ruby, it limits its functionality to only managing Ruby versions. However, in our case, we want to install Ruby as well.

Let's modify the rbenv.rake file and add a new run_installer task:

# lib/capistrano/tasks/rbenv.rake

namespace :rbenv do
  desc "Map Ruby binaries to use rbenv"
  task :map_bins do
    ...
  end

  desc "Change shell startup file to setup rbenv"
  task :modify_shell_file do
    ...
  end

  desc "Install rbenv and rbenv-build tools" 
  task :run_installer do
  end
end

Similar to the task for modifying a shell startup file, we want to skip installing rbenv if it's already setup. To do so, we're going to use bash shell to check for rbenv directory existence:

desc "Install rbenv and rbenv-build tools"
task :run_installer do
  on release_roles(:all) do
    next if test("[ -d #{rbenv_path} ]")
  end
end

We start by saving the rbenv installer location in a method rbenv_installer_url:

def rbenv_installer_url
  "https://github.com/rbenv/rbenv-installer/raw/master/bin/rbenv-installer"
end

One word of caution here. Before using a third-party script, it's best to read through and verify that a script doesn't contain any malicious code first. For extra measure, it's advisable to store such script on an internal network and only download it from there.

Then, installing rbenv is just a matter of downloading the installation script using the wget utility in quiet mode. Once the installation script is downloaded, we're going to pipe its content into the Bash shell to perform the installation process:

desc "Install rbenv and rbenv-build tools"
task :run_installer do
  on release_roles(:all) do
    next if test(" [ -d #{rbenv_path} ] ")

    execute :wget, "-q", rbenv_installer_url, "-O-", "|", :bash
  end
end

And with that we have successfully installed rbenv utility on our server. This way we have opened the doors for automating Ruby installation.

Summary

In this article, we wrote a few fairly complex tasks that gave us a much bigger taste of Capistrano capabilities. We modified environment variables, updated a shell startup file and installed rbenv utility. As we did all this, we explored more of the Unix shell concepts and learnt how rbenv works.

As a side benefit, this article demonstrated to us the extreme flexibility that Capistrano architecture gives us. It lets you provision the application infrastructure the way you need. You describe your requirements using task definitions and, viola! Not sure about you, but I'm impressed with Capistrano design. It embraces Ruby's dynamic nature and object model to provide you with an intuitive way of expressing deployment concepts.

Nice. We’re another level up on our path to Capistrano proficiency. In the next article, we will continue to explore how to combine all the tasks to install a brand-new Ruby.

This article was originally published on PiotrMurach.com.

Photo by Tincho Franco on Unsplash