Getting Started with Puppeteer & Headless Chrome for End-to-End Testing

Inspiration for this post

This blog post was inspired by a talk I saw at the Connect.Tech Conference in Atlanta this year, entitled: "Using Puppeteer to Write End-to-End Tests That Run On The Browser."

The speaker, Taylor Krusen, gave a great talk on Puppeteer and headless Chrome and using them together to take end-to-end testing to the next level. I’d looked briefly into Puppeteer in the past, but since I couldn’t find any great examples of how to get started with it, I’d passed and chosen to use Cypress.io for my testing needs instead.

After the talk though, I was inspired to give Puppeteer another shot and I went a step further, figuring out how to run Puppeteer and how to use it with Jest, the testing framework most commonly associated with React.

And I want to share that today to hopefully help others get started with their own application testing suites.

What is end-to-end testing & why is it important?

End to end tests make up the smallest part of the testing pyramid, as defined by Google, but they are still a crucial part of it.

Before I dive into how to use Puppeteer for end-to-end testing, I think it’s important to briefly explain what it is and why it’s important for web development today.

Techopedia gives a good definition of end-to-end testing:

End-to-end testing is a methodology used to test whether the flow of an application is performing as designed from start to finish. The purpose of carrying out end-to-end tests is to identify system dependencies and to ensure that the right information is passed between various system components and systems.

Essentially, end-to-end testing replicates, in an automated fashion, exactly what a user could do while interacting with the application or browser. And this is done to ensure the application works properly and as expected.

As web applications get more and more complex and full featured, it becomes harder and harder for manual QA teams to keep up with testing every single piece of functionality each time the application is ready to deploy a new feature. End-to-end testing frameworks were created to take the burden (and repetitive task) of checking functionality out of the hands of people and have machines (which can move much faster than actual users and exactly replicate a test every time) shoulder more of the load.

Ultimately, it still comes down to us, as developers, to ensure our applications work the way we expect, but testing helps us to better ensure we’re not introducing new bugs into our code or compromising existing functionality, which is why it’s so important.

What are Headless Chrome & Puppeteer?

The next thing to cover is what exactly are Headless Chrome and Puppeteer, and why are they always bundled together?

Puppeteer’s GitHub and npm pages do a great job of explaining exactly what Puppeteer is:

Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol.

It goes on to say Puppeteer runs headless Chrome or Chromium instances by default, which is why they’re always mentioned in tandem.

Most things that you can do manually in the browser can be done using Puppeteer!

Puppeteer was made for Chrome by the Google Chrome DevTools team to help further automated/headless browser testing and make it less painful for the rest of us.

The biggest drawback is that Puppeteer only works with Chrome. I have had the good fortune of working on applications where our users use Chrome exclusively, which made Puppeteer a very good option for end-to-end testing, but if the user base is more varied in its browser preferences, it may be wise to look into other testing frameworks like Cypress.io, TestCafe, or Selenium Web Driver.

But today, at least, I get to talk about Puppeteer.

Puppeteer examples

Now it’s time to get down to the business of getting Puppeteer up and running.

Getting started with Puppeteer is actually incredibly easy — just type npm i puppeteer and you’re ready to go. Yes, it really is that simple. It’s this easy because the Chrome DevTools team conveniently packages each version of Puppeteer with a version of Chromium (the headless Chrome part) that it’s guaranteed to run with.

NOTE: Puppeteer requires at least Node v6.4.0, but the examples below use async/await which is only supported in Node v7.6.0 or greater. Just so ya know.

Let’s take a look at what Puppeteer can automate.

If you’d like to play with these examples yourself, you can download my GitHub repo here.

You can also click on the file name for each of these code samples below to see the actual code in GitHub.

Take a screenshot of a page: saveFileExample.js

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.google.com');
  await page.screenshot({ path: 'google.png' });

  await browser.close();
})();

As you’ll see with all these scripts, as long as puppeteer is imported at the top of the file, it’s good to go and all the methods packaged along with Puppeteer like newPage() and browser are instantly available to you.

This first sample script goes to the Google home page and takes a photo of the page and saves it to a .png file inside of the repo named "google.png".

The code is pretty self-explanatory. To begin, a headless Chrome browser is launched, a new page is opened, the URL for that page is set to Google, a screenshot is taken for posterity and the browser is closed. The whole thing is clean, elegant, asynchronous and wrapped within an IIFE (immediately invoked function expression) so as soon as the script is called, the function runs.

Create a PDF: createPDF.js

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.thinkgeek.com/', { waitUntil: 'networkidle2' });
  await page.pdf({ path: 'thinkGeek.pdf', format: 'A4' });

  await browser.close();
})();

The second Puppeteer example goes to a website (the fun retail website, ThinkGeek, in this case) and makes the site page it visits into a PDF.

Similar to the first script, this one sets up Puppeteer, creates a new browser page, goes to ThinkGeek’s website and then uses the page.pdf() method to make a PDF of the whole page. That PDF is once again, saved into the project folder under the name "thinkGeek.pdf". Pretty simple.

Google something: searchGoogle.js

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://www.google.com');

  const searchBox = await page.$('input[type=text]');
  await searchBox.type('cookies');
  const inputElement = await page.$(
    'input[type=submit][value="Google Search"]',
  );
  await inputElement.click();
  const [response] = await Promise.all([
    page.waitForNavigation(),
    page.once('load', () => console.log('Cookies loaded!')),
  ]);

  await page.screenshot({ path: 'cookies.png' });
  await browser.close();
})();

This third example is probably something more like what would be tested in an end-to-end test — it’s automating a Google search (and taking a screenshot to confirm the search worked, in headless mode).

In this script, Puppeteer launches a new browser page, it goes to Google, it waits for the search box and the search button to be visible and types "cookies" into the input, then it waits for the page to navigate to the cookies search results and takes a screenshot before closing the browser at the end.

And that’s all there is to it. To see more of what Puppeteer can do, I recommend checking out their very thorough documentation — there’s a whole lot of info and functionality to discover there.

Debugging Puppeteer

Now that Puppeteer is running, I want to call out some awesome tools that are built into it to make debugging it much less painful. Here's the code snippet you need, which I'll break down next.

const browser = await puppeteer.launch({
  headless: false,
  slowMo: 500,
  devtools: true
})

Turn off headless mode

The first (and most obvious one) is how to turn off Chrome’s headless mode; meaning you can watch the automated browser open and attempt to go through the steps laid out for it.

This is a cinch. Inside the script where Puppeteer is launched (pretty much the first thing that happens when the script runs), just add the following line: puppeteer.launch({headless: false});.

Enable SlowMo

SlowMo is just what it sounds like, it slows down Puppeteer’s operations so normal humans can see what’s going on. It can be instantiated in the same command where headless is set to false for Puppeteer; the puppeteer.launch() command. Just add puppeteer.launch({slowMo: 500}) or however many milliseconds you want to slow down the actions by.

Auto-open DevTools panel

The third debugging trick I have is how to tell Puppeteer whether to auto-open a Chrome DevTools panel with each tab. So if you need to, you can catch things being logged to the console inside the automated test run.

Once again, inside the initial command launching Puppeteer, add in puppeteer.launch({headless: false, devtools: true}); to open a DevTools panel.

NOTE: If this option is true, the headless option will be set false.

You can even console.log() events from the headless Chrome browser to your own terminal by simply throwing this method from page.exposeFunction() into the mix. Right after the page object is defined in the Puppeteer script, you can do:

page.on("console", msg => console.log(`Page Console: ${msg.text()}`));

to catch messages from console.log.

That should help get you on your way to debugging your Puppeteer headless Chrome instances and tests like a pro.

More cool features of Puppeteer

In addition to debugging Puppeteer, I wanted to point out a few more cool features that it offers. If you’re interested in learning more about any of them, I’d recommend checking out their npm page for more info.

Puppeteer can:

• Run in a Docker container or serverless environment,
• Intercept network requests,
• Capture performance info,
• Test a Chrome extension,
• Run code in a page, and
• Emulate Chrome on a mobile device.

That’s a lot of additional useful things it can be used for. The team really wants to make it useful to devs in many ways.

Bonus: React & Jest & Puppeteer

I’d be remiss if I didn’t also briefly mention how to integrate Puppeteer into a React project with Jest tests. React is my current JavaScript framework of choice (and it seems the same for many JS developers), so I created a quick setup and test script using the two together, also housed in the my sample GitHub repo here.

Configuration for Jest & Puppeteer

Jest has some very helpful documentation to smoothly work Puppeteer into its own testing framework, and there’s even an entire npm package named jest-puppeteer to help the process along. Here’s what to install and configure to make Jest and Puppeteer play nice.

npm install --save-dev jest-puppeteer puppeteer jest 

Once everything’s been installed, you can either configure your Jest config file, Webpack file or your package.json file to let Jest know that Puppeteer’s there and ready to be used. Here’s how I did it in the package.json.

Right above the scripts in the file, I added this snippet, and inside the scripts, I included a test command which just runs Jest.

package.json

"jest": {
    "preset": "jest-puppeteer"
  },
  "scripts": {
    "test": "jest",
    /* more scripts here */
  }

Now, I’m ready to write tests.

Jest & Puppeteer Test: google.test.js

describe('Google', () => {
  beforeAll(async () => {
    await page.goto('https://google.com');
  });

  it('should display "google" text on page', async () => {
    await expect(page).toMatch('google');
  });
});

If you’ve ever written a Jest test before, the syntax should look familiar. Since I don’t have a full React project as part of my sample repo, I’ll use this simple example test where Jest should go to the Google homepage and find the text "Google” on said page after it’s loaded.

From my repo, I set up the npm script to run the jest command when npm run test is entered into the terminal. Jest looks for any files with the *.test.js syntax, by default and tries to run those. When it finds my test file entitled google.test.js it runs it (headlessly, unless I were to specify differently) and logs out the test results to the console, like so.

If tests failed, you can debug just as you normally would (and even turn on the automated Chrome browser if you need to see what’s messing up), and if not, your computer screen can be test free as your end-to-end tests still run in the background.

Awesome. That doesn’t seem so hard, now does it?

Conclusion

Puppeteer is a powerful tool developed by the Google DevTools team to make headless, automated browser testing easier in every way. There’s less configuration to get started, the syntax makes more sense, and it has uses that extend far beyond just testing web apps during development. Likewise, it’s easy to integrate into React applications to automate Jest tests to run headlessly as well.

Check back in a few weeks, I’ll be writing about something else related to web development.

If you’d like to make sure you never miss an article I write, sign up for my newsletter here: https://paigeniedringhaus.substack.com

Thanks for reading, I hope this gives you an idea of how to get started using Puppeteer for your own end-to-end testing and integrating with React and Jest tests.

References & Further Resources

• Puppeteer Jest sample repo on GitHub
• End-to-end testing on Techopedia
• Puppeteer docs on npm
• Puppeteer repo on GitHub
• Puppeteer API docs on GitHub
• Jest with Puppeteer site docs