When I ask developers how they test their applications with UI usually the answers are related to unit-testing or to testing some specific components. I know that some of the managers even add a particular value of the code coverage as a requirement. They hope it will reduce amount potential problems and the code will be more reliable with coverage near to 100%. However, the truth is not that pleasant.
Unit tests don’t test application behavior that sees end users and doesn’t prevent all the bugs
If you want to be sure that the application is working as it was written on requirements you might consider verification of particular user scenarios. For me, a user scenario – is the described step-by-step user behavior in the application. Simply speaking, requirements one or another way describes how to achieve user goals by using the application. So when I’m checking a specific function in the unit test it’s not the same for sure, because the user works with the system on a much higher level.
What to do in this case? The answer is – writing the tests that will check requirements, not only classes and functions, I mean acceptance or e2e tests. In my case requirements are described in the form of the user scenarios with Gherkin and I will write acceptance tests
For you, it might be written a different way but the main idea is the same and I will show you how easy it can be for you.
Getting started
I’m going to use Cypress(version 12.2.0) for writing the tests for my UI and I don’t want to set up the whole backend and deal with creating specific users for testing purposes. I will use these tests in the pipeline of my front-end project. So the installation is pretty simple
npm i --save-dev cypress
We need to run cypress the first time to have all the config files created, so let’s do that
cypress open
It will open the interactive cypress application that will guide you and create the config and cypress folder in the repository. If you haven’t used cypress before I highly recommend accepting the suggestion from cypress about creating examples of the tests.suggestion from cypress about creating examples of the tests.
This way you will have a lot of examples that will help in the initial step.
Start writing tests
We have plenty of examples produced by cypress, but let’s do something on our own, to check how difficult it is. I’ve chosen npm.com UI and autocomplete on this page. But I’m not going to just check the autocomplete work on this page, but isolate UI and mock API. Let’s add one more file npm.autocomplete.cy.js
in the e2e folder and write our test
describe('NPM tests', () => {
beforeEach(() => {
cy.visit('https://www.npmjs.com/');
});
it('Npm autocomplete test', () => {
cy.intercept('GET', 'https://www.npmjs.com/search/suggestions?q=step', [
{
name: 'step',
scope: 'unscoped',
version: '1.0.0',
description: 'Test',
date: '2017-01-06T21:16:57.341Z',
links: { npm: 'https://www.npmjs.com/package/step' },
author: { name: 'John Smith', email: 'test@test.com', username: 'test' },
publisher: { username: 'test', email: 'test@test.com' },
maintainers: [{ username: 'test@test.com', email: 'test@test.com' }],
},
]).as('getResults');
cy.get('input[name="q"]').type('step');
cy.wait('@getResults');
cy.get('ul[role="listbox"] li').should('have.length', 1).should('have.text', 'stepTest1.0.0');
});
});
First of all, I’ve added describe
block and added cy.visit
in beforeEach
function, which will open npm.com before every test that we have below. The test itself is in it
block and in the beginning, we’re intercepting the request that the page will do when we type something in the search field. Pay attention that we’ve defined the alias getResults
Next, cy.get
is used to select the input field and type the word "step" there and cy.wait
will help us to wait for our mocked response. Now it’s time to check what we have in the search results with should
function and everything look correct.
We haven’t reviewed the features of the cypress here but has wonderful documentation with a huge amount of examples, so it won’t be a big challenge. Probably remember that I have requirements in the Gherkin form, so let’s make it possible to use Gherkin for writing tests.
Testing Gherkin
Cypress has a lot of officially supported plugins and I’m going to use one of them cypress-cucumber-preprocessor (version 15.0.0) for Gherkin support.
npm i --save-dev @badeball/cypress-cucumber-preprocessor
Additionally, I will install @bahmutov/cypress-esbuild-preprocessor
(version 2.1.5) preprocessor, because it has no particular requirements and the config is the smallest.
Let’s update the cypress.config.ts
with the new configuration. Pay attention I switched to TypeScript, it’s a pretty easy move with cypress because it’s described in the documentation. Don’t forget to add tsconfig.json
if you go this way. Anyway it’s optional, so if you don’t need it – stay with the JavaScript.
This is the config, after the updates:
import { defineConfig } from "cypress";
import createBundler from "@bahmutov/cypress-esbuild-preprocessor";
import { addCucumberPreprocessorPlugin } from "@badeball/cypress-cucumber-preprocessor";
import createEsbuildPlugin from "@badeball/cypress-cucumber-preprocessor/esbuild";
export default defineConfig({
e2e: {
specPattern: "**/*.feature",
async setupNodeEvents(
on: Cypress.PluginEvents,
config: Cypress.PluginConfigOptions
): Promise<Cypress.PluginConfigOptions> {
// This is required for the preprocessor to be able to generate JSON reports after each run, and more,
await addCucumberPreprocessorPlugin(on, config);
on(
"file:preprocessor",
createBundler({
plugins: [createEsbuildPlugin(config)],
})
);
// Make sure to return the config object as it might have been modified by the plugin.
return config;
},
},
});
Now we’re ready to re-write our test to Gherkin, so let’s create a .autocomplete.feature
file and add the following scenario there.
Feature: NPM search
Scenario: Searching the package
Given I've opened the npm.com
When I typed a phrase in the search field
Then I should see the results in the dropdown menu
If you run tests after this step they will fail, because we haven’t defined the steps definitions, so it’s time to define it. By default, you can place steps definitions wherever you want in the e2e
folder, but if you had many tests it would be a mess. I use subfolders within the e2e
so for our case it will be e2e/autocomplete/autocomplete.ts
import { When, Then, Given } from '@badeball/cypress-cucumber-preprocessor';
Given(`I've opened the npm.com`, () => {
cy.visit('https://www.npmjs.com/');
cy.intercept('GET', 'https://www.npmjs.com/search/suggestions?q=step', [
{
name: 'step',
scope: 'unscoped',
version: '1.0.0',
description: 'Test',
date: '2017-01-06T21:16:57.341Z',
links: { npm: 'https://www.npmjs.com/package/step' },
author: { name: 'John Smith', email: 'test@test.com', username: 'test' },
publisher: { username: 'test', email: 'test@test.com' },
maintainers: [{ username: 'test@test.com', email: 'test@test.com' }],
},
]).as('getResults');
});
When('I typed a phrase in the search field', () => {
cy.get('input[name="q"]').type('step');
});
Then('I should see the results in the dropdown menu', () => {
cy.wait('@getResults');
cy.get('ul[role="listbox"] li').should('have.length', 1).should('have.text', 'stepTest1.0.0');
});
As you can see we’ve split our test into several steps, but it works the same way.
One of the benefits of such splitting is the possibility to re-use the steps. In this case, we’ve added one more scenario that starts with the same step we don’t need to define the Given
once again in the scope of this feature. So, in the end, we write less code and some of the tests might be literally written in .feature
files.
Scenario: No results is present for invalid value
Given I've opened the npm.com
When I typed incorrect search phrase
Then I see no results
Additional configuration and reporting
The most important thing about the tests is to make it to help you in your day-by-day work. For this, it has to be easy to use and it should be integrated with the process of development. Let’s add a couple of scripts in our package.json
"scripts": {
"cy:open": "cypress open",
"cy:run": "cypress run"
},
cy:open
– is opening the interactive mode for writing and debugging the tests
cy:run
– will run all the tests and generate the report
You will say here “Wait a moment, have we configured reporting?”. You’re goddamn right, we didn’t, but we will fix it in a moment by adding one more section to package.json
Here is my config:
"cypress-cucumber-preprocessor": {
"messages": {
"enabled": true,
"output": "report/cucumber-messages.ndjson"
},
"html": {
"enabled": true,
"output": "report/cucumber-messages.html"
},
"json": {
"enabled": false
}
}
You can read a bit more about configuring the cypress-cucumber-preprocessor
in the documentation, if you need to customize something.
Ok, let’s try to run the tests and see what we have
In the real project, I have a similar report for every merge request and it’s deployed separately for the main branch, so the management team can easily check what scenarios are covered by the test and what’s missing.
Conclusion
As you can see adding acceptance tests for testing your UI is not a difficult task and can be introduced relatively quickly. Of course, you need to cover at least the most important user scenarios to make such tests useful. The main goal of it is to catch issues as early as possible. Then in case, your changes broke something it will have been noticed immediately. It would be much better if you found the issue not on the production on Friday, right?!
I haven’t considered the cypress itself in detail in this article, because it wasn’t the goal. However, I can give you a couple of hints I’ve found after practicing it for some time:
- use the cypress commands for the things that are used often in the code. I mean some standard selectors, creating mocks or interceptors. It’s strictly specific to the project, so I can’t tell you what exactly to put there, but you will find out yourself, after writing several tests
- don’t use classes or ids for selecting UI elements, use special data attributes for it. It will prevent situations of breaking the tests accident by changes in the styles.
- don’t create scenarios that are checking only the UI or some specific labels. Acceptance and e2e tests are expensive and if you test everything there it will take significant time not only for writing and supporting tests but for execution. You can use an integration test instead to cover this part.
- Gherkin is a good option, but it’s not necessary. If your team doesn’t have requirements written in Gherkin – you can start without it. It’s much better to have less expressive tests than have no tests at all
Thank you for reading and good luck with testing!
Top comments (0)