A Rails 6 system test is a test that exercises your application in a way that, as much as possible, simulates a real user interacting with it via a browser. More than any other kind of tests, system tests verify that the whole app does what it’s supposed to do.

As such, they are typically used for acceptance testing: tests that verify that an app has fully implemented a specified feature. They are also useful for smoke tests: high-level tests that exercise the code just enough to verify that nothing is badly broken. As well as characterization tests: tests that capture an app’s current behavior in preparation for internal refactoring.

Technically, a system test doesn’t have to interact with an actual browser. They can be configured to use the rack_test backend, which simulates HTTP requests and parses the resulting HTML. While rack_test-based system tests run faster and more reliably than frontend tests using a real browser, they are significantly limited in their ability to simulate a real user’s experience, because they can’t run JavaScript.

For the rest of this article I’m going to be focusing on system tests that interact with your app via an actual browser.

There are a lot of moving parts involved in running a browser-based system test. Until recently, I was pretty fuzzy on how all these parts fit together. Recently I set out to get browser-based system tests running for a Dockerized Rails app, and as a result I’ve learned a bit more about them.

At a high level, a Rails system test putting an app through its paces using a browser breaks down as:

• A MiniTest test case, augmented with…
• Capybara testing helpers, which
  • start and stop an instance of your app, and
  • provide an English-like DSL on top of…
• The selenium-webdriver gem, which provides a Ruby API for using the…
• … WebDriver protocol in order to interact with…
• A WebDriver tool such as chromedriver or geckodriver , which…
• Is automatically downloaded by the webdrivers gem. The WebDriver tool automates…
• A browser , such as Chrome.

The Geology of a System Test

Let’s look at each of those in more detail.

MiniTest

MiniTest is the unit testing library that Rails’ built-in testing framework is based on. It provides base classes for test cases, basic assertions like assert_equal, and a runner to run tests and report on their success and failure.

For Rails System Tests, Rails provides an ApplicationSystemTestCase base class which is in turn based on ActionDispatch::SystemTestCase:

require "test_helper"
​
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :selenium, using: :chrome, screen_size: [1400, 1400]
end
​

This base class augments your system test cases with Capybara abilities.

Capybara

Capybara is a Ruby library for full-stack testing of web applications. It provides your tests with a few different capabilities.

First, Capybara arranges for an instance of your app to be started before the tests start executing, and shuts it down once the tests have finished. In other words, for a Rails app it does the equivalent of running rails server before executing the tests, and hitting Ctrl-C once they are done. To keep it separate and to ensure that the tests are run against the right test instance, Capybara starts your app on a different port than the one you’d normally use to manually interact it.

Second, Capybara provides a high-level, English-style API for authoring system tests. Capybara’s DSL is what enables you to write your tests in terms similar to how you’d describe the interactions to a human:

visit "/"
click_on "Log in"
fill_in "Username", with: "avdi"
fill_in "Password", with: "xyzzy"
click_on "Submit"
# ...

In a sense, Capybara acts as a kind of stable pivot point for around which all the other aspects of testing a Ruby web application can vary. Capybara is neutral towards:

• The web app framework you’re building on. Whether you’re building on Rails, Sinatra, Hanami, or raw Rack, you can test it with the help of Capybara.
• The test tooling you prefer. Whether your tests are written in MiniTest, RSpec, or Cucumber, they can all incorporate the Capybara helper methods for controlling a simulated browser.
• The type of browser simulation/automation. Capybara lets you switch out “drivers” to automate different kinds of browsers, real or simulated. The same Capybara tests can be run against bare Rack requests, actual Chrome, Safari, or Firefox browsers; “headless” versions of those browsers, or simulated browsers such as PhantomJS.

When Capybara is automating a real browser, it does so by translating calls to its the high-level Capybara DSL (click_on, fill_in) into invocations of a library called selenium-webdriver.

selenium-webdriver

Automatic control of a browser is typically accomplished using a WebDriver tool, which is a standalone executable for remote-controlling a particular browser. We’ll talk more about WebDriver executables in a minute.

WebDriver executables all speak the WebDriver protocol, an HTTP-based protocol for automating browsers. Capybara doesn’t speak WebDriver directly. Instead, it makes use of a library called selenium-webdriver.

What’s with the name “selenium”? The Selenium project is the umbrella under which the WebDriver protocol for controlling browsers was first developed. This protocol is now a W3C standards track spec. The selenium project still maintains various language bindings to the WebDriver protocol. The selenium-webdriver gem contains the Ruby webdriver bindings.

Here’s an example of using selenium-webdriver directly, that I swiped from the project documentation:

require 'selenium-webdriver'
​
driver = Selenium::WebDriver.for :firefox
wait = Selenium::WebDriver::Wait.new(timeout: 10)
​
begin
  driver.get 'https://google.com/ncr'
  driver.find_element(name: 'q').send_keys 'cheese', :return
  first_result = wait.until { driver.find_element(css: 'h3>div') }
  puts first_result.attribute('textContent')
ensure
  driver.quit
end

You can see how it’s a bit lower-level than the Capybara example further up. The selenium-webdriver library translates these calls into WebDriver Protocol, which it speaks to a webdriver executable.

WebDriver Protocol

WebDriver is an HTTP-based protocol for telling browsers what to do. For instance, in order to create a Chrome browser window and tell it to navigate to google.com, you would first start up a chromedriver process. Then, you’d send it a “new session” command with an HTTP post:

curl -X POST 'http://127.0.0.1:9515/session' -d '{"capabilities":{"firstMatch":[{"browserName":"chrome"}]}}'

This returns a bunch of data, including a session ID:

{ ... "sessionId":"98de922f9be6784957775f25618c60a3" ... }

Which we follow up with a “navigate to” command in the form of POSTing a new url to the created session:

curl -X POST 'http://127.0.0.1:9515/session/97039faaa396941517d800b75d0f6096/url' -d '{"url": "https://google.com"}'

This is the level of tedium that the selenium-webdriver gem is abstracting over with its Ruby bindings.

You can’t send these WebDriver commands directly to a browser, however. You have to send them to a WebDriver process.

WebDriver Tool

“WebDriver” is a protocol. A WebDriver, on the other hand, refers to a tool that speaks that protocol and controls a browser. If you want to drive a browser with WebDriver protocol, you need to first download that browser’s WebDriver tool.

WebDriver tools act as servers: when you execute them, they start a persistent process that listens for HTTP requests until it is terminated.

For every major browser, there is an associated WebDriver. Chrome has chromedriver. Firefox has geckodriver. MS Edge has edgedriver. Safari has safaridriver.

While they speak a common language, WebDriver tools aren’t all developed by the same people. Each browser’s WebDriver is a separate project, usually associated with the browser’s development team. Each WebDriver’s team typically provides binaries for every major platform that browser is available on.

Finding, downloading, and installing the appropriate WebDriver tool somewhere in your PATH is a hassle. That’s why Rails 6 projects, by default, use the webdrivers gem to automatically download an up-to-date WebDriver tool that corresponds to Capybara’s configuration.

The webdrivers gem

This part of the stack is really just a convenience. When the webdrivers gem is required, it automatically augments the selenium-webdriver gem. Now when you run selenium-webdriver tests, it automatically determines which WebDriver executable needs to be downloaded for your platform and selected browser, downloads it, and arranges for that executable to be used by selenium-webdriver.

With the WebDriver tool downloaded, your tests can drive a browser!

Browser

Once you have a WebDriver tool installed, the parts farther up the stack can use it to automate browser interactions. As system tests are run, you’ll see new browser windows appear, navigate to pages in your app, fill in forms, follow links, etc.

Conclusion

It takes a lot of layers to make a Rails 6 system test work! There are a lot of moving parts, and a lot of opportunities for things to go wrong. Personally, I’ve spent a quite a bit of time fiddling with system test configuration in order to get them running consistently. Once they are running though, it’s a particularly satisfying feeling to watch your app being put through its paces automatically.

Once you’ve managed to get system tests running in real browser windows locally, you can move on to the next step of getting them to run “headless”. Then you can have a continuous integration server such as CircleCI automatically run those slow-but-important system tests for you after every commit.

But that’s a topic for another article!