DEV Community

Cover image for Planning Your Next Cypress.io Tests with Testomat.io
davert
davert

Posted on

Planning Your Next Cypress.io Tests with Testomat.io

It is so easy to start writing tests with Cypress.io. It seems that Cypress opened doors to testing for everyone with basic JavaScript knowledge. However, when the number of tests grows writing new tests becomes harder. And the issue is not just about writing code. Planning new test scenarios, refactoring, configuring pipelines, and getting test run reports. The more people involved in the product development the more visibility the testing process needs.

To keep end 2 end tests organized, following questions should be answered:

  • what tests do we have?
  • what parts of an application are covered with tests?
  • what new test scenarios should be implemented?
  • what is the current state for passed and failed tests?
  • if there are flaky tests on a project?

🤔

Cypress.io is just a test runner, but it can be paired with Testomat.io, a test management system for automated tests.
Let's see how Testomat.io can answer the questions risen above. We will use Cypress Realworld App, which is a real-world project (as you might have guessed) with a well-written Cypress tests.

Creating project in Testomat.io

🤔 What tests do we have?

But answering the very first question is not as simple as it seems.

All Cypress.io end 2 end tests are written as JavaScript code so to read them a person should be at least familiar with JavaScript, GitHub, etc. So this already clothes doors for business analysts, product managers, and manual QAs, for all these folks who do not actually code.

That's why we can import a Cypress realworld app into a newly created project on Testomat.io.

Import project from source code in Testomat.io

Importing happens with a magical npx check-tests command which scans all the tests on a project and imports them into UI without executing them. This way Testomat.io can present those tests to non-tech folks.

To import only UI tests from realworld app project, we need to run check-tests command with --typescript flag (as the project is written in TypeScript) specifying that we use Cypress.io and the path to tests via --dir parameter:

TESTOMATIO=a9ae6798atoj npx check-tests@latest Cypress.io "**/*{.,_}{test,spec}.ts" --dir cypress/tests
Enter fullscreen mode Exit fullscreen mode

a9ae6798atoj is an API key of a project you are importing to, so in your case, it will be a different one.

If you use Import from Source button, you will get this command generated. Take note, that by default it imports tests from cypress/integration folder which doesn't exist in cypress-realworld project. So you need to change path and use cypress/tests instead.

Importing is fast, it takes just a few seconds to get this output.

Import output of Cypress.io project in Testomat.io

Then all imported tests can be reviewed.

Listed Cypress.io tests in Testomat.io

That's great that all tests are finally visible to team members. But is there a way they could be linked to specifications they are testing? Yes, and that answers the next question:

🤔 What parts of an application are covered with tests?

If the development management happens in Jira, Testomatio can link automated tests to Jira issue. You can link tests from a Cypress spec to an Issue:

Linking Test to Jira Issue in Testomat.io

Now all tests will be available within Jira, so can be taken into the development process. Revealing automated tests to Jira Issues helps managers and business analysts to understand more information on quality control. They don't need to leave Jira to see those automated tests!

Automated Tests in Jira Testomat.io Plugin

The only thing needed, is a Testomatio Jira plugin should be installed from a marketplace and a Jira Project to be connected to Testomatio project.

And we are almost ready to answer the next question:

🤔 What new test scenarios should be implemented?

A project manager or a business analyst knows what features are coming next so they are the right person to propose the next tests.

As they can't be asked to write a code, or open an issue for each test, they could create in Testomat.io test directly from Jira. After that, a developer can take the written scenario and implement it using Cypress.io.

Writing Tests inside Jira Testomat.io plugin

As you see on the screenshot, step autocompletion can be used to help non-tech folks write scenarios efficiently.

The test with a description is treated as a manual test, but once it is implemented as Cypress test, and re-imported it will be marked as automated.

🤔 What is the current state for passed and failed tests

Testomat.io can display and store Cypress reports. To send run reports to Testomat.io a @testomatio/reporter package should be installed and enabled in cypress/plugins/index.ts.

// inside cypress/plugins/index.ts
// require testomatio reporter package:
const testomatioReporter = require('@testomatio/reporter/lib/adapter/cypress-plugin');


export default (on, config) => {
  testomatioReporter(on, config);

  // ....
Enter fullscreen mode Exit fullscreen mode

This reporter can be added to any Cypress.io project, following instructions from Testomat.io

Setting up reporting in Testomat.io

Then tests should be executed via npx cypress run with few extra parameters:

TESTOMATIO=a9ae6798atoj npx cypress run
Enter fullscreen mode Exit fullscreen mode

a9ae6798atoj is an API key of a project to which reports will be sent.

A report will contain a list of passed and failed tests, with videos and screenshots. External S3 storage from one of the cloud providers (AWS, DigitalOcean, Google Cloud, etc) can be used to keep unlimited screenshots and videos.

Unlimited Test Artifacts for Cypress.io Tests in Testomat.io

Videos and screenshots can be enabled in Testomatio by adding S3 credentials into .env file of Cypress Realworld project:

S3_ACCESS_KEY_ID=aws-key
S3_SECRET_ACCESS_KEY=aws-secret
S3_REGION=ap-south-1
S3_BUCKET=testomatio-test
Enter fullscreen mode Exit fullscreen mode

A complete historical data of all Cypress.io test executions are also available:

Run Reports in Testomat.io

If we have this data the final question can be answered:

🤔 Are there flaky tests on a project?

Testomat.io has an analytics dashboard where all tests are analyzed to provide valuable insights. For instance, it is important to keep track of flaky tests, and slowest tests, and check for the most common defects across tests.

Flaky Tests Detection in Testomat.io

Final Thoughts

Developing end-2-end tests with Cypress.io is fun. But making them first-class citizens in project development is also important. Cypress End 2 end tests must be visible to the whole team, so even team members who are not directly involved in testing could track the progress, propose new changes to tests, and see the actual test coverage.

Testing is a crucial part of the development process. The more clear it is and the more team members are aware of it, the more quality software can be delivered. While Cypress.io helps build efficient and fast tests for the modern web, Testomat.io provides a feature-rich test management solution for all kinds of automated tests.

P.S. Testomat.io is absolutely free to use for personal projects. Try it out with Cypress realworld app, or any other Cypress project.

Top comments (3)

Collapse
 
liviufromendtest profile image
Liviu Lupei

Cypress isn't really popular these days, mostly due to the limitations.

Such limitations include:

  1. Not being able to run tests on Safari (see What Happens When You Don't Test in Safari)
  2. Not being able to run tests in multiple browser tabs (see How to test in multiple tabs)
  3. No full support for iframes
Collapse
 
davert profile image
davert

I agree on the limitations. They are a lot of them!
However, Cypress fits well niche of e2e an component testing for modern SPA

For more complete End2End testing experience I would look into Playwright and CodeceptJS

Collapse
 
liviufromendtest profile image
Liviu Lupei

Those libraries work great for folks looking to build a career out of writing tests.

Most developers I talked to just want to create the tests as fast as possible and be done with them, that's why they end up using platforms like Endtest or Mabl