This post has originally been published on https://juristr.com/blog/2019/07/testing-cypress-intro. Go to juristr.com/blog for more content
Writing automated tests has become like a standard practice in software development and (should be) an integral part of a healthy software development process. Testing (in particular automated testing) can come with different flavors and at different levels. In this article I'd like to provide a quick overview of automated UI or end-2-end testing with Cypress.
(from my newsletter. Go subscribe here)
As important as testing is, it is also a question of cost vs. benefit. We don't want to blindly test everything, but the most critical parts and create tests that give us most benefit (which we can also call ROI - Return on Investment). First of all, let's talk about manual vs. automated testing. I often think about it this way:
- Manual testing is spending money (and you need to do that over and over again)
- Automated testing is investing money which over time comes with a ROI (i.e. hopefully a less buggy app 😉)
Manually testing isn't investing. It's spending money to get a one-time feedback, that's it. Automated tests give us continuous feedback over time. And that Return on Investment (ROI) of your tests is exactly what we want to strive for.
But even with automated tests, we have different ones. At a high level, I like to classify them between unit level and integration level tests. And this is also often when you see the mythical testing pyramid:
The testing pyramid told us which tests to write based on the speed of execution and the cost of writing them. And while integration level tests were told to be slow & costly, they give us the most value. If you think about it, they are much nearer to what our end-user would do, while unit level tests are more of a developer tool, verifying how the independent pieces work in isolation.
The recent changes in the available tooling in the web development space started to challenge this view a bit. In fact, Kent C. Dodds started to provide an alternative way of the testing pyramid, what he calls "the testing trophy".
Testing with Cypress
If we focus more on the UI (and e2e) testing area, there are a couple of tools around. Most of them are based on the Selenium driver. I've been using them in combination with Java, .Net and more successfully with wrappers like NightwatchJS and Protractor. Selenium is great, it provides an abstraction layer around the browser APIs that allows you to write selenium tests in a variety of different programming languages. However, that very same abstraction makes it often also cumbersome and verbose to use, and hard to debug.
Cypress is a relatively new tool on the marktetand that is definitely worth giving a look. Not only does it feel extremely refreshing to use it, with a simple API, but it also has a couple of cool killer features, such as time travel, improved debugging, real-time reloads and automatic waiting.
With Cypress you write integration level tests via browser automation. Frankly, it clicks the buttons you tell it to click and verifies whether the resulting state matches our expectations. In that point it is very similar to Selenium. And still, it's quite different. When talking about automated UI testing I usually get a stomachache. I've been using lots of those automation tools and mostly they resulted in flaky tests and tons of false negatives..to the point where you just throw them away or disable them. Cypress seems to have learned from that and does an awesome job on dealing with the async nature of the web, providing top-notch debugging capabilities and perfect integration with your webapp. One of the core differences is that Cypress runs "inside" your browser, just alongside your web app.
As a result it has access to the DOM as well as all ongoing XHR requests, giving us even the possibility to stub them out. This opens up a lot of possibilities for an improved testing experience. But before I go into too much details, take a look at my 15min lightning talk I gave about a year ago.
Also, don't forget to take a look at the Cypress docs. The team has invested a ton of time to make them great and approachable.
Top comments (9)
Thank you for the interesting post. Altought, I feel like the "test trophy" is more like a gimmick than a real thing. "Static testing" (eslint, types and stuff) is inherent to the test pyramid as well.
To me it's not clear why people want to find excuses to avoid writing real 1x1 unit tests (the base of the pyramid).
The only plausable reason to avoid unit tests is if the codebase is already rotten (aka: legacy and untested), then you might go with an inverted pyramid just to keep regression under control.
It's not like the pyramid cannot be criticized. But it is a good pattern to follow in 95% of the cases. And one might take care not to demonize it for those who are yet learning about test automation, code design and tdd. :)
Hey, thx for your comment. Yea for sure, unit tests are still extremely valuable. But if you re-think for a sec the whole testing scenario, those tests that give you the most value are those that integrate the entire thing, right. To simulate the user basically.
But then of course there is also developer experience (like how easy are those tests to write, how maintainable, how fast are they on CI etc...).
There's of course no silver bullet, but Cypress addresses a lot of the issues why previously people somehow tried to avoid those slow, costly, hard to maintain end-to-end (UI based) tests :)
I second to what Zizaco said - both in thanking and defending the Test Pyramid, especially when it comes to new learners for whom Test Pyramid is currently part of de-facto methodology.
Having said that, I do not think that mentioning stubbing capabilities of Cypress aligns well with the test trophy suggested approach, used as the keynote to the article.
This is good article but if you have provided some insights like cypress,handling web pages and how does it interacts with xhr along with request that would be great. Let me know what you think. Stuff looks good but too generic
Hey, you're totally right. In fact this is just a high level intro to Cypress for getting started. I could indeed write a more in depth blog post about how the whole XHR mocking and stuff works. I'll put it on my backlog. Subscribe in case to my newsletter to get notified (juristr.com/newsletter/).
Otherwise you could always check out the official Cypress docs which are excellent: docs.cypress.io/guides/overview/wh...
Cypress looks great! I think I'll give it a try today... Have you run into any edge cases where it just really doesn't work well?
Thanks for the post and the talk.
Hey, not directly. You have to pay attention to keep them fast as that's what mostly matters. They have a good best practices list to follow here: docs.cypress.io/guides/references/...
Other than that, it's really easy to write them and they have been super valuable so far.
Ah - thanks for the extra info!
Some comments may only be visible to logged-in visitors. Sign in to view all comments.