Playwright is a powerful web testing tool supporting Chromium, Firefox and WebKit engines. The project has excellent documentation and many examples.
I use Playwright for E2E tests on the DEV environment. Today I'm going to show how you can integrate Playwright on GitHub Actions more efficiently.
The documentation suggests using the following snippet to run Playwright inside GitHub Actions:
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run your tests
run: npx playwright test
- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: playwright-report
What is notable here?
- It's supposed to run all tests on the same type of GitHub Runner.
ubuntu-latest
by default. - All supported browser engines (chromium, firefox, WebKit) will be installed on the Linux runner.
- Playwright will execute tests for all browser engines from the same runner.
- The last step will prepare an archive with an HTML report.
After several months of experiments I come up with the following configuration:
# ...
jobs:
test:
name: π§ͺ ${{ matrix.project }} E2E Tests
runs-on: ${{ matrix.os }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
- project: chromium
os: ubuntu-latest
cache_dir: ~/.cache/ms-playwright
- project: firefox
os: ubuntu-latest
cache_dir: ~/.cache/ms-playwright
- project: webkit
os: macos-12
cache_dir: ~/Library/Caches/ms-playwright
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version-file: '.nvmrc'
cache: 'npm'
- name: β‘οΈ Cache NPM dependencies
uses: actions/cache@v3
id: cache-primes
with:
path: node_modules
key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}
- name: β‘οΈ Cache playwright binaries
uses: actions/cache@v3
id: playwright-cache
with:
path: ${{ matrix.cache_dir }}
key: ${{ runner.os }}-${{ matrix.project }}-pw-${{ hashFiles('**/.playwright-version') }}
- name: π₯ Install deps
if: steps.cache-primes.outputs.cache-hit != 'true'
run: make install
- name: π₯ Install ${{ matrix.project }} with Playwright
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps ${{ matrix.project }}
- name: π Playwright tests
run: npx playwright test --project=${{ matrix.project }}
env:
DEBUG: pw:api,pw:browser*
- name: π Upload test results
if: failure()
uses: actions/upload-artifact@v3
with:
name: playwright-report-${{ matrix.project }}
path: playwright-report
What I've changed in GitHub Workflow for Playwright tests?
- I use a matrix with three browser engines to run the jobs simultaneously. Instead of
ubuntu-latest
, I run WebKit tests onmacos-12
runner. I had to make the change to avoid fragile tests. I noticed that thepage.goto()
API works differently on Linux and macOS for the WebKit engine. WebKit didn't wait forload
event, which causes failures from time to time. At the same time, I've never caught such a problem locally on my MacBook. If you're interested, the ticket is still open in the Playwright bug tracker. - The
fail-fast
parameter is set to false to wait for all testing jobs to complete, even if one of them fails for some reason. - I believe it is essential to specify
timeout-minutes
for E2E tests so that the tests will crash if the infrastructure hangs or behaves incorrectly. - I use the .nvmrc file inside the repository to specify the NodeJS version. DRY all over the place!
- Configured aggressive cache for npm dependencies and binary files with browsers. I added a special target to Makefile to calculate the Playwright browsers hash:
bump-playwright:
npm install --save-dev @playwright/test
npx playwright --version > .github/.playwright-version
After executing make bump-playwright
command, the latest PW version will be installed and the value of the installed release will be written to the .github/.playwright-version file. GitHub Actions calculate the hash for the cache from the file content. Example file:
$ cat .github/.playwright-version
Version 1.29.2
- Each runner installs only one browser and it's pretty handy cause the HTML report will have the results for that browser β easy to navigate and spot the issues. This distinction also allows you to have an idea of the test execution speed in each browser and to notice flickering tests specific to a particular engine.
- Playwright tests run with enabled DEBUG, so that in case of a crash, you can quickly get additional information for diagnostics.
- In contrast to the recommended configuration from the documentation, I prefer to upload artifacts with the report only in case of test failures. I don't see the point in storing those artifacts on the cloud, given how often the tests are run.
I shared my findings for configuring E2E playwright tests in GitHub Actions. Feel free to ask questions about the example GitHub workflow.
What Playwright tricks do you use in your daily work? Would love to hear about your hidden gems π.
Cover photo by Denny MΓΌller.
Top comments (1)
Brilliant examples and explanations! Thx!