Playwright is a remarkable tool that lets you author cross-browser cross-platform end-to-end tests for your web app. It even lets you generate test code as you navigate your app (but that's probably a topic for another blog post)!

Unfortunately, as of now, out of the box, it does not support measuring code coverage for the app being tested (see #7030 and #9208). This post demonstrates a workaround.

Specifically, this post shows you how to instrument a Next.js app using a custom babel.config.js and how to measure code coverage using the playwright-test-coverage plugin. If you want to jump straight into the demo, you can do that here:

npm install
npm test

Step 1: Instrument your code

If you want to measure code coverage, first you will want to instrument your code (i.e. insert counters without causing any side-effects) using istanbul. Most modern web apps (React, Vue etc), use babel in their toolchain (which converts modern ES2015+ code into backwards compatible code to support older browsers). The recommended way to instrument code in this scenario is using babel-plugin-istanbul:

npm i -D babel-plugin-istanbul

Now, we need to instruct babel to use this plugin. The details may vary by framework, but for a Next.js app, create a file named babel.config.js in the root folder:

// babel.config.js
const plugins = [];

// Instrument for code coverage in development mode
if (process.env.NODE_ENV === "development") {
  console.log(
    "Detected development environment. Instrumenting code for coverage."
  );
  plugins.push("istanbul");
}

module.exports = {
  presets: ["next/babel"],
  plugins,
};

This will instrument code only in development mode e.g. when running next dev. That's good because we do not want instrumentation cluttering our production code.

Step 2: Write tests using the playwright-test-coverage extension

Now that your code is instrumented, you are ready to measure code coverage. For this you will need to install this Playwright plugin:

npm i -D playwright-test-coverage

This package exports overrides for Playwright's test and expect methods which handle code coverage measurement correctly.

// tests/app.spec.js
const { test, expect } = require("playwright-test-coverage");

test("basic test", async ({ page }) => {
  // Use Playwright as usual
  await page.goto("http://localhost:3000");
  await expect(title).toHaveText("Title");

  // ...
});

If you have existing Playwright tests, simply drop in the following replacement near the top:

// Replace this:
const { test, expect } = require("@playwright/test");

// With this:
const { test, expect } = require("playwright-test-coverage");

Step 3: Measure code coverage

Finally, you are ready to measure code coverage.

Start the Next.js dev server using the Istanbul command line tool nyc - which enables code coverage measurement:

npx nyc next dev

Then, in another terminal window, run the playwright tests as usual:

npx playwright test

If all goes well, you should see a code coverage report similar to:

Running 1 test using 1 worker

  ✓  tests/app.spec.js:5:1 › basic test (1s)


  1 passed (2s)
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |
 index.js |     100 |      100 |     100 |     100 |
----------|---------|----------|---------|---------|-------------------

Wrap up

Hopefully you now have some understanding of how to measure code coverage for a Next.js (or any Babel based) app when running Playwright tests.

You can check out the full source code for the demo here:

npm install
npm test

The extension code is available here and contributions/issues are welcome:

npm i -D @playwright/test babel-plugin-istanbul nyc

npm i -D playwright-test-coverage

// tests/foo.spec.js
const { test, expect } = require("playwright-test-coverage");

// Use test and expect as usual
test("basic test", async ({ page }) => {
  await page.goto("https://playwright.dev/");
  const title = page.locator(".navbar__inner .navbar__title");
  await expect(title).toHaveText("Playwright");
});

Cheers!

PS: Using with fixtures

Playwright lets you define custom fixtures in accordance with the Page Object Model pattern. This library should work out of the box with custom fixtures.

You can see an example here.