Moving your Rails test suite from PhantomJS to Headless Chrome

This post was originally written in 2018. Some of the details may have changed since then, but I didn't want this information to go to waste, so take from what it what you will.

A Rails app I worked on had long relied on PhantomJS for its Capybara and Teaspoon tests, and with great appreciation. It's provided an excellent way to ensure users are receiving a consistent, battle-tested experience. It regularly runs in excess of 800 integration and JavaScript tests.

That's why it was super exciting to hear that Chrome was shipping a fully-functional headless mode, because as great as PhantomJS is, it was severely lacking in many modern browser features and standards*. With Chrome we could now more accurately test to reflect what our users were actually seeing, and not have to worry about polyfills and vendor prefixes just to satisfy the test suite.

But the fact that you're here probably means you already knew that. So I'm going to break down for you how this Rails app migrated its test suite from PhantomJS to headless Chrome.

^{* But we still greatly appreciate the effort and dedication that went in to maintaining PhantomJS. Thank you Vitaly and others!}

Preparation

Before you jump in I'd recommend getting familiar with Chrome from the command line. Here's a good primer.

It goes without saying that, like any app upgrade, you should have a green test suite before starting. If you know you've got some particularly intricate tests, make a note to ensure they're still behaving the way they should afterward.

If you're using CI, ensure it supports ChromeDriver. This Rails app uses Semaphore, which provides support out of the box.

You'll also need ChromeDriver on your machine to run tests locally. Installation comes in a couple flavours:

# via NPM
npm install chromedriver --global
# via Brew
brew cask "chromedriver"

Finally, your Rails app is going to need Selenium and the ChromeDriver helper. Update your Gemfile's test group accordingly:

group :test do
+  gem 'capybara-selenium'
+  gem 'chromedriver-helper'
-  gem 'poltergeist'
end

Getting behind the wheel

Now that you have the prerequisites you can start to swap out your Poltergeist drivers with Selenium ones. Selenium with Chrome accepts quite a number of arguments, so I'll break down a few key ones:

• headless - this is an obvious one, in that it enables Chrome to operate in headless mode. However, since we have a full-blown Chrome browser at our disposal it might be wise to create a driver that doesn't include this argument, for cases where you want to watch things happen.
• window-size - this might also seem obvious, but I would always advise explicitly setting the window dimensions. Otherwise you might do what I did and assume it defaulted to some large desktop size and spend hours wondering why certain elements weren't visible.
• blink-settings=imagesEnabled=false - indeed, this takes us in to Chrome's rendering engine, Blink, and disables image rendering. This helps with page speed.
• disable-gpu - while this argument was once required across the board, it is now only required to fix bugs present on Windows systems.

With these in mind, let's register a driver:

default_chrome_options = %w(
  --disable-extensions
  --disable-gpu
  --disable-infobars
  --disable-notifications
  --disable-popup-blocking
  --ignore-certificate-errors
  --incognito
  --mute-audio
  --remote-debugging-port=9222
)

# Some additional arguments you might consider
#
# --disable-default-apps
# --disable-password-generation
# --disable-password-manager-reauthentication
# --disable-password-separated-signin-flow
# --disable-save-password-bubble
# --disable-translate
# --no-default-browser-check
# --proxy-server=host:port
# --start-maximized (or --kiosk)

# Go through each argument and add it to an options instance
chrome_options = Selenium::WebDriver::Chrome::Options.new
default_chrome_options.each { |o| chrome_options.add_argument(o) }

# This will allow us to view console logs; more on this below
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
  loggingPrefs: {
    browser: "ALL",
    client: "ALL",
    driver: "ALL",
    server: "ALL"
  }
)

# Instantiate our HTTP client
client = Selenium::WebDriver::Remote::Http::Default.new(open_timeout: nil, read_timeout: 120)

# Finally, register the driver itself
Capybara.register_driver :headless do |app|
  chrome_options.add_argument("--headless")
  chrome_options.add_argument("--blink-settings=imagesEnabled=false")
  chrome_options.add_argument("--window-size=1280,1024")

  Capybara::Selenium::Driver.new(
    app,
    browser: :chrome,
    desired_capabilities: capabilities,
    clear_local_storage: true,
    clear_session_storage: true,
    options: chrome_options,
    http_client: client
  )
end

You'll notice that we've started with an array of standard arguments, and then added a few more inside the driver registration. That's because our app actually registers a handful of other drivers and we want them all to inherit the same same standard set of options. By no means do you need to use any or all of these arguments; go ahead and experiment with what works best for your setup!

I'll note one other neat feature Chrome introduces: device emulation. Previously if you wanted to imitate a mobile device, you might do something like:

page.driver.headers['User-Agent'] = "Mozilla/5.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/600.1.3 (KHTML, like Gecko) Version/8.0 Mobile/12A4345d Safari/600.1.4"
page.driver.resize(375, 667)

Now you can simply register a driver using the add_emulation option:

chrome_options.add_emulation(device_name: 'iPhone 8')

Here's the [gross looking] full list of devices.

You can even get more specific about the characteristics of your emulation, with arguments like pixelRatio and touch. Read more about it here.

Lastly, make sure your tests use the drivers:

# test_helper.rb
Capybara.javascript_driver = :headless

# teaspoon_env.rb
config.driver = :selenium
config.driver_options = { client_driver: :chrome }

What's different in Chrome?

el.trigger does not exist

We've all been there. Your test can't find an element that you're trying to .click.

Maybe it's something funky with your CSS, or maybe PhantomJS isn't rendering it correctly. A common workaround was to trigger(:click). Not anymore! Selenium doesn't implement the .trigger method.

Thankfully, since Chrome is much more up to date and closer to rendering what the actual experience will look like, you should be able to reliably .click your elements. If you're struggling to figure out why something isn't receiving an event this is a good time to bust out your GUI Chrome driver to see exactly what's going on.

You'll need to accept alerts yourself

While before you could rely on PhantomJS to automatically accept confirm dialogues, with Chrome you need to do it yourself. It's as simple as:

# Before
click_link 'Do the thing!'

# After
accept_confirm { click_link 'Do the thing!' }

# Lazy? Put it in a helper method
def click_link_and_accept(text)
  accept_alert do
    click_link text
  end
end

def click_button_and_accept(text)
  accept_alert do
    click_button text
  end
end

Resizing the window has changed

If you're resizing the window you're going to need to update how it's called:

# Before
page.driver.resize(1024, 600)

# After
page.driver.browser.manage.window.resize_to(1024, 600)

Keyboard events have changed

If you're sending keyboard events to an element using el.native.send_keys, the main differences in Chrome are that

1. The element you're sending keys to needs to be focusable
2. If you're sending a symbol, things are slightly but annoyingly different. For example, :Right is now :right

fill_in doesn't fire a change event

While we're talking about keyboard events, with PhantomJS fill_in("#something", with: "I like turtles") would automatically fire a JavaScript change event. That no longer happens in Chrome. To get that event you have a couple options:

# Add a newline to `with`
fill_in("#something", with: "I like turtles\n")

# Do it in JavaScript
page.execute_script("document.querySelector('#something').dispatchEvent(new Event('change'))")
# Or jQuery if you're a millennial
page.execute_script("$('#something').trigger('change')")

Both of those solutions are ripe to be turned in to helper methods as well.

You'll need to dig deeper for logs

PhantomJS would automatically print console logs to the terminal. With Chrome you're going to need to do a little more work.

First, when setting up your driver make sure you're setting up the loggingPrefs capability like we did above.

Then, if you need to see a console output you can access it like so:

# :key can be :browser, :client, :driver, :server
page.driver.manage.get_log(:key)

But wait! There's more:

We're using Chrome, so let's take full advantage of it and just use DevTools!

You may have seen the driver argument --remote-debugging-port=9222 above. By using that any time you're running your tests you can then head over to http://localhost:9222 and inspect elements, view the console, and pretty much everything else you would expect from DevTools. 😍

Other nice-to-haves

• Ever used save_and_open_page? The cool kids are just using the GUI Chrome driver. If you need to pause execution, just throw a byebug in there and you'll then be able to play around in the Chrome browser.
• If you need to clear your browser cache on the fly, here's a handy method:

  def clear_browser_cache
    return unless page.mode.to_s.include?("chrome")
    page.driver.visit('chrome://settings/clearBrowserData')
    find("* /deep/ #clearBrowsingDataConfirm").click
  end

Your mileage may vary

While these steps should get you most of the way there, I'm willing to bet that you're still going to have some test failures. That's okay. Spend some time with it, because it's absolutely worth it.

I'll note that most of what I uncovered in migrating this Rails app's test suite to Chrome was cobbled together from a variety of resources, so I thank these pioneers greatly, and I highly recommended checking out their posts for further reading:

• How GitLab switched to Headless Chrome for testing
• Gist: translating old capybara selenium/chrome preferences and switches
• Moving From PhantomJS to Headless Chrome in Rails: Fewer Hacks, Simpler Debugging and More Consistent Tests
• Switching to Headless Chrome for Rails System Tests
• Class: Selenium::WebDriver::Chrome::Options

I hope that this breakdown helps you on your way to better client-side testing, because it's certainly helped us. If you've got tips or feedback based on your own experience, please do leave a comment below.