DEV Community

Akbar Nafisa
Akbar Nafisa

Posted on

Improving Design System Visual Testing Consistency with Docker

#webdev #javascript #vue #testing

Visual Testing is a great way to test the consistency of your UI component in a Design System. However, it can become an issue when working on a project with multiple developers and trying to run the test either locally or in CI, as the UI may appear different for each developer. Therefore, it is necessary to run the tests on the same browser and environment to ensure consistency.

Docker

Docker is an open-source platform that enables developers to create, deploy, and run applications inside isolated containers. By using Docker, we can ensure consistent results across different environments and browser version.

Implementation

This project was inspired by an awesome design system that implemented their own visual testing, which inspired me to explore how it works. You can see the full implementation for this project in this GitHub repository. Now, let's discuss how to implement it.

Create UI Component

This project uses Lerna to manage its packages

- packages
    - core // Core component
    - design-tokens // Tokens
    - visual-test // UI Testing
Enter fullscreen mode Exit fullscreen mode
  • core package: this is the core component of the Design System. We use Vue 3, Vite, and TypeScript. We also use Storybook to display the component use cases and Jest for unit testing.
  • design-tokenks package: We generate the tokens from this package.
  • visual-testing package: We test the UI components from this package.

Create Visual Test

This package utilizes several libraries

  • jest: We use it to create our test cases.
  • jest-html-reporter: We create a custom reporter to show which UI tests fail.
  • jest-image-snapshot: We use this library to capture the UI screen and check changes between tests.
  • jest-puppeteer-docker: We use this library to run the Puppeteer in the Docker.
  • puppeteer: We use this library to interact with the browser to open the page to check for visual changes.

Prerequisites

Before we run the test, we need to install Docker. The easiest way to do it is by installing Docker Desktop. We also need to build the Storybook from the core package so that we can use it for visual testing.

Setup Jest

We set the Jest configuration to support the Visual test.

module.exports = {
  preset: 'jest-puppeteer-docker',
  // specify a list of setup files to be executed after the test framework has been set up but before any test suites are run.
  setupFilesAfterEnv: ['./jest-setup/test-environment-setup.js'],
  // executed once before any test suites are run
  globalSetup: './jest-setup/setup.js',
  // The function will be triggered once after all test suites
  globalTeardown: './jest-setup/teardown.js',
  testMatch: ['**/?(*.)+(visual.spec).[tj]s?(x)'],
  modulePathIgnorePatterns: ['<rootDir>/dist/'],
    // add jest-html-reporter to be our costume reporters
  reporters: [
    'default',
    [
      'jest-html-reporter',
      {
        outputPath: './visual-test-result/index.html',
        pageTitle: 'Test Result',
        includeFailureMsg: true,
        // Path to a javascript file that should be injected into the test report,
        customScriptPath: './inject-fail-images.js',
      },
    ],
  ],
};
Enter fullscreen mode Exit fullscreen mode

Then, we add the setup file to the initialization of the HTTP server from the build file of Storybook.

const { setup: setupPuppeteer } = require('jest-puppeteer-docker');
const path = require('path');
const http = require('http');
const fs = require('fs');

const server = http.createServer((req, res) => {
  // parse URL
  const url = new URL(req.url, `http://${req.headers.host}`);

  // serve static files from "static" directory
  const filePath = path.join(__dirname, '../storybook-static', url.pathname);
  fs.readFile(filePath, (err, data) => {
    if (err) {
      // if file not found, return 404 error
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.write('404 Not Found');
      res.end();
    } else {
      // if file found, return file contents
      res.writeHead(200, { 'Content-Type': getContentType(filePath) });
      res.write(data);
      res.end();
    }
  });
});

// helper function to get content type based on file extension
function getContentType(filePath) {
  const extname = path.extname(filePath);
  switch (extname) {
    case '.html':
      return 'text/html';
    case '.css':
      return 'text/css';
    case '.js':
      return 'text/javascript';
    case '.json':
      return 'application/json';
    case '.png':
      return 'image/png';
    case '.jpg':
    case '.jpeg':
      return 'image/jpeg';
    default:
      return 'application/octet-stream';
  }
}

module.exports = async (jestConfig) => {
  // start server on port 3000
  global.__SERVER__ = server.listen(3000, () => {
    console.log('Server started on port 3000');
  });

  await setupPuppeteer(jestConfig);
};
Enter fullscreen mode Exit fullscreen mode

Then, we add a teardown configuration. In this configuration, we close the HTTP server and copy the test results to the reporter directory file. For more information about setup and teardown configuration, you can read the documentation of jest-puppeteer-docker.

const fs = require('fs');
const fse = require('fs-extra');
const path = require('path');

const { teardown: teardownPuppeteer } = require('jest-puppeteer-docker');

module.exports = async function globalTeardown(jestConfig) {
  global.__SERVER__.close();
  await teardownPuppeteer(jestConfig);

  const dirname = path.join(__dirname, '..');

  fs.copyFileSync(
    `${dirname}/jest-reporters/inject-fail-images.js`,
    `${dirname}/visual-test-result/inject-fail-images.js`
  );

  try {
    fse.copySync(
      `${dirname}/src/__image_snapshots__/__diff_output__`,
      `${dirname}/visual-test-result/__diff_output__`,
      {
        overwrite: true,
      }
    );
  } catch {}
};
Enter fullscreen mode Exit fullscreen mode

Next, we add jest-puppeteer-docker config. See another config here.

const getConfig = require('jest-puppeteer-docker/lib/config');

const baseConfig = getConfig();
const customConfig = Object.assign(
  {
    connect: {
      defaultViewport: {
        width: 1040,
        height: 768,
      },
    },
    browserContext: 'incognito',
    chromiumFlags: '–ignore-certificate-errors',
  },
  baseConfig
);

module.exports = customConfig;
Enter fullscreen mode Exit fullscreen mode

Finally, we add a global function to navigate to the Storybook page and another global function to do visual test.

const { toMatchImageSnapshot } = require('jest-image-snapshot');

jest.setTimeout(10000);

expect.extend({ toMatchImageSnapshot });

global.goto = async (id) => {
  await global.page.goto(
    `http://host.docker.internal:3000/iframe.html?id=${id}&viewMode=story`
  );
  await page.waitForNavigation({
    waitUntil: 'networkidle0',
  });
};

global.testUI = async () => {
  await global.page.waitForSelector('#root');
  const previewHtml = await global.page.$('body');
  expect(await previewHtml.screenshot()).toMatchImageSnapshot();
};
Enter fullscreen mode Exit fullscreen mode

Setup Jest Reporter

After we finish testing, we usually want to check the test results. If there are any failed tests, we can display the comparison result here packages/visual-test/visual-test-result/.

document.addEventListener('DOMContentLoaded', () => {
  [...document.querySelectorAll('.failureMsg')].forEach((fail, i) => {
    const imagePath = `__diff_output__/${
      (fail.textContent.split('__diff_output__/')[1] || '').split('png')[0]
    }png`;

    if (imagePath) {
      const div = document.createElement('div');
      div.style = 'margin-top: 16px';

      const a = document.createElement('a');
      a.href = `${imagePath}`;
      a.target = '_blank';

      const img = document.createElement('img');
      img.src = `${imagePath}`;
      img.style = 'width: 100%';

      a.appendChild(img);
      div.appendChild(a);
      fail.appendChild(div);
    }
  });
});
Enter fullscreen mode Exit fullscreen mode

Adding tests

To add new tests, usually it’s easier to take a screenshot of each Storybook page. Take a look at the example below, where we have a button component with 5 different pages.

Image description

Then, we add the test case for each page.

describe('Button', () => {
  test.each([['variants'], ['size'], ['disabled'], ['full-width']])(
    '%p',
    async (variant) => {
      await global.goto(`buttons-button--${variant}`);
      await global.page.evaluateHandle(`document.querySelector(".c-button")`);
      await global.testUI();
    }
  );
});
Enter fullscreen mode Exit fullscreen mode

You can also use Puppeteer API if you want to take a screenshot after specific element action.

Running Test

To run the tests you can follow these steps:

  • Build Storybook by using yarn workspace @contra-ui/vue build-storybook --quiet
  • Copy the Storybook build file from the core package to visual-test by using yarn workspace @contra-ui/visual-test copy
  • Open your Docker desktop and run the test by using yarn workspace @contra-ui/visual-test test. If you run the test for the first time, it will take a while for Docker to download the image and build the container.
  • Check the test result in packages/visual-test/src/image_snapshots to see the screenshot if it’s expected

Checking the failing tests

To check the failed test you can open the HTML report file in packages/visual-test/visual-test-result/.

Image description

Jest will place the differences in a folder, which you can inspect at packages/visual-test/src/__image_snapshots__/__diff_output__/. To update the test, you can add the -u flag: yarn workspace @contra-ui/visual-test test

CI Setup

Next, let’s move the step that we have done locally to CI. There are three additional steps that are needed to automatically create PR to update the failed tests.

name: build-pr
on:
  pull_request:
    branches:
      - main

jobs:
  install-dependency:
    name: Install depedency
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Install Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 16
          cache: 'yarn'
      - name: Install dependencies
        run: yarn --immutable

  visual-tests:
    name: Visual tests
    needs: [install-dependency]
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Install Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 16
          cache: 'yarn'

      - name: Install dependencies
        run: yarn --immutable

      - name: Run Lerna bootstrap
        run: yarn lerna:bootstrap

      - name: Build Storybook
        run: yarn workspace @contra-ui/vue build-storybook --quiet

      - name: Copy Storybook for visual tests
        run: yarn workspace @contra-ui/visual-test copy

      - name: Run visual tests
        run: yarn workspace @contra-ui/visual-test test -u

      - name: Check for any snapshots changes
        run: sh scripts/porcelain.sh

      - name: Set patch branch name
        if: failure()
        id: vars
        run: echo ::set-output name=branch-name::"visual-snapshots/${{ github.head_ref }}"

      - name: Create pull request with new snapshots
        if: failure()
        uses: peter-evans/create-pull-request@v4
        with:
          commit-message: 'test(visual): update snapshots'
          title: 'update visual snapshots: ${{ github.event.pull_request.title }}'
          body: This is an auto-generated PR with visual snapshot updates for \#${{ github.event.number }}.
          labels: automated pr
          branch: ${{ steps.vars.outputs.branch-name }}
          base: ${{ github.head_ref }}
Enter fullscreen mode Exit fullscreen mode

  • Step name: Check for any snapshots changes
    In this step, we will check if there is a new commit. Because we set -u flag, it will automatically modify the existing file. When we find any modified file, the script exits with a non-zero exit code.

    echo "--------"
echo "Checking for uncommitted build outputs..."
if [ -z "$(git status --porcelain)" ];
then
    echo "Working copy is clean"
else
    echo "Another Pull Request has been created. Please check it to accept or reject the visual changes."
    git status
    exit 1
fi

  • Step name: Set patch branch name
    In this step, we will save the branch name as an output variable if there are any previous steps have failed.

  • Step name: Create pull request with new snapshots
    In this step, we will create a pull request for the failed branch, this step will make it easier for us to see what UI changed and we can accept the changes if it’s expected, here is the pull request example:

You also need to update your workflow permissions on the Setting page to enable the GitHub Action to create Pull Request.

Image description

Wrapping it up

The usage of docker can be really used fully to make our visual tests more consistent, both locally and in CI. I hope this post has given you some ideas. You can see the full code in this GitHub repository.

Top comments (0)

Read next

grjoeay profile image

How AI Accelerates Test Automation Capabilities

Joe Gray -

mayank_mohapatra profile image

Creating a Multilingual Expense Tracker with Tolgee

Mayank Mohapatra -

im_santhiya_guna profile image

Strapi — Why

Santhiya G -

undolog profile image

Upgrading from Nextra v2 to Nextra v3: A "Quick" Guide

Giovambattista Fazioli -