Rails System Tests In Docker

Originally posted on Hint's blog.

At Hint, we use Docker extensively. It is our development environment for all of our projects. On a recent greenfield project, we wanted to use Rails System Tests to validate the system from end to end.

In order to comfortably use System Tests inside of Docker we had to ask ourselves a few questions:

1. How do we use RSpec for System Tests?
2. How do we run the tests in headless mode in a modern browser?
3. Can we run the test in a non-headless browser for building and debugging efficiently?

Context

In the context of answering these questions, we are going to first need to profile the Rails project to which these answers apply. Our Rails app:

• Uses Docker and Docker Compose.
• Uses RSpec for general testing.
• Has an entrypoint or startup script.

If some or none of the above apply to your app and you would like to learn more about our approach to Docker, take a look at this post: Dockerizing a Rails Project.

Prep Work

If your project was started before Rails 5.1, some codebase preparation is necessary. Beginning in Rails 5.1, the Rails core team integrated Capybara meaning Rails now properly handles all the painful parts of full system tests, e.g. database rollback. This means tools like database_cleaner are no longer needed. If it is present, remove database_cleaner from your Gemfile and remove any related config (typically found in spec/support/database_cleaner.rb , spec/spec_helper.rb , or spec/rails_helper.rb).

Dependency Installation

After that codebase prep has been completed, verify that RSpec ≥ 3.7 and the required system tests helper gems are installed.

group :development, :test do
  gem 'rspec-rails', '>= 3.7'
end

group :test do
  gem 'capybara', '>= 2.15'
  gem 'selenium-webdriver'
  # gem 'chromedriver-helper' don't leave this cruft in your Gemfile.:D
end

These are the default system test helper gems installed with rails new after Rails 5.1. We will not need chromedriver-helper since we will be using a separate container for headless Chrome with chromedriver.

Note: The configuration below has been tested on Mac and Linux, but not Windows.

Docker

Speaking of containers, let's add that service to the docker-compose.yml configuration file.

services:
  # other services...
  selenium:
    image: selenium/standalone-chrome

We have added the selenium service, which pulls down the latest selenium/standalone-chrome image. You may notice I have not mapped a port to the host. This service is for inter-container communication, so there is no reason to map a port. We will need to add an environment variable to the service (app in this case) that Rails/RSpec will be running on for setting a portion of the Selenium URL.

services:
  app:
    build: .
    command: bundle exec rails server -p 3000 -b '0.0.0.0'
    # ... more config ...
    ports:
      - "3000:3000"
      - "43447:43447"
    # ... more config ...
    environment:
      - SELENIUM_REMOTE_HOST=selenium

Capybara

We added a port mapping for Capybara as well: 43447:43447. Now let's add the Capybara config at spec/support/capybara.rb.

# frozen_string_literal: true

RSpec.configure do |config|
  headless = ENV.fetch('HEADLESS', true) != 'false'

  config.before(:each, type: :system) do
    driven_by :rack_test
  end

  config.before :each, type: :system, js: true do
    url = if headless
            "http://#{ENV['SELENIUM_REMOTE_HOST']}:4444/wd/hub"
          else
            'http://host.docker.internal:9515'
          end

    driven_by :selenium, using: :chrome, options: {
      browser:              :remote,
      url:                  url,
      desired_capabilities: :chrome
    }

    # Find Docker IP address
    Capybara.server_host = if headless
                             `/sbin/ip route|awk '/scope/ { print $9 }'`.strip
                           else
                             '0.0.0.0'
                           end
    Capybara.server_port = '43447'
    session_server       = Capybara.current_session.server
    Capybara.app_host    = "http://#{session_server.host}:#{session_server.port}"
  end

  config.after :each, type: :system, js: true do
    page.driver.browser.manage.logs.get(:browser).each do |log|
      case log.message
        when /This page includes a password or credit card input in a non-secure context/
          # Ignore this warning in tests
          next
        else
          message = "[#{log.level}] #{log.message}"
          raise message
      end
    end
  end
end

Let's break it down section by section.

headless = ENV.fetch('HEADLESS', true) != 'false'

We are using the headless variable to make some decisions later in the file. This variable allows us to run bundle exec rspec normally and run system tests against headless Chrome in the selenium container. Or we run HEADLESS=false bundle exec rspec and when a system test will attempt to connect to chromedriver running on the host machine.

config.before :each, type: :system do
  driven_by :rack_test
end

Our default driver for system tests will be rack_test. It is the fastest driver available because it does not involve starting up a browser. It also means we cannot test JavaScript while using it, which brings us to the next section.

config.before :each, type: :system, js: true do
  url = if headless
          "http://#{ENV['SELENIUM_REMOTE_HOST']}:4444/wd/hub"
        else
          'http://host.docker.internal:9515'
        end

  driven_by :selenium, using: :chrome, options: {
    browser:              :remote,
    url:                  url,
    desired_capabilities: :chrome
  }

  # ...more config...       
end

Any specs with js: true set will use this config. We set the url for Selenium to use depending on if we are running headless or not. Notice the special Docker domain we are setting the non-headless url to; it is a URL that points to the host machine. The special domain is currently only available on Mac and Windows, so we will need to handle that for Linux later.

We set our driver to :selenium with config options for browser, url, desired_capabilities .

config.before :each, type: :system, js: true do

  # ...selenium config...

  Capybara.server_host = if headless
                           `/sbin/ip route|awk '/scope/ { print $9 }'`.strip
                         else
                           '0.0.0.0'
                         end
  Capybara.server_port = '43447'
  session_server       = Capybara.current_session.server
  Capybara.app_host    = "http://#{session_server.host}:#{session_server.port}"
end

Here we set the Capybara.server_host address to the app container IP address if headless or 0.0.0.0 if not.

The last part of RSpec configuration is to require this config in spec/rails_helper.rb .

# frozen_string_literal: true

# This file is copied to spec/ when you run 'rails generate rspec:install'
require 'spec_helper'
ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
# Prevent database truncation if the environment is production
abort("The Rails environment is running in production mode!") if Rails.env.production?
require 'rspec/rails'
require 'support/capybara'

Next, we need to install ChromeDriver on the host machine. You will need to place it in a location in your $PATH. Once it is there, when you want to run non-headless system tests, you will need to start ChromeDriver chromedriver --whitelisted-ips in a new terminal session. Now on a Mac, you should be able to run headless or non-headless system tests. Those commands again are:

#HEADLESS
bundle exec rspec

#NON-HEADLESS
HEADLESS=false bundle exec rspec

Special Linux Config

There is one last step for Linux users because of the special host.docker.internal URL is not available. We need to add some config to the entrypoint or startup script to solve that issue.

: ${HOST_DOMAIN:="host.docker.internal"}
function check_host { ping -q -c1 $HOST_DOMAIN > /dev/null 2>&1; }

# check if the docker host is running on mac or windows
if ! check_host; then
  HOST_IP=$(ip route | awk 'NR==1 {print $3}')
  echo "$HOST_IP $HOST_DOMAIN" >> /etc/hosts
fi

We set an environment variable to the special Docker URL. We then create a function to check if the host responds to that URL. If it responds, we move on assuming we are running on Mac or Windows. If it does not respond, we assign the container's IP to an environment variable, then append a record to /etc/hosts. We are now all set to run system tests on Linux as well.

Bonus: CI Setup

Let's wrap this up with config to run system tests on Circle CI. We need to add SELENIUM_REMOTE_HOST and the Selenium Docker image to .circleci/config.yml

version: 2
jobs:
  build:
    parallelism: 1
    docker:
      - image: circleci/ruby:2.6.0-node
        environment:
          SELENIUM_REMOTE_HOST: localhost
      - image: selenium/standalone-chrome

Connect with me on Twitter(@natron99) to continue the conversation about Rails and Docker!