The power of End-to-End tests

In this article, I will advocate for using End-to-End (E2E) tests. I will start by defining what E2E tests are and what they are not. After this, I will try to convince you to use them as one of the essential bricks of your future and current projects. The last section is providing answers to the most common criticisms addressed to E2E tests.

Definition

E2E tests ensure that an entire system verifies a set of requirements. There are two implications to that definition:

1. A set of requirements is defined.
2. We have the ability to run an entire system in our testing setup.

E2E tests are written in a way that we can verify that the requirements are fulfilled by the system. They can be as simple as "Users can sign into the application using the Login form" and as complex as describing a sequence of actions that users will likely go through while using the system.

What a good E2E testing setup should not do:

• Mock parts of the application, especially the critical ones, like the database and the ORM (Object Relational Mapping). Ideally, a E2E testing setup doesn't mock anything.
• Tests should not describe how the system works, rather focus on how users interact with the system.

Why use E2E tests?

I see two main reasons why every project should have a strong emphasis on E2E tests:

1. They guarantee that requirements are fulfilled. They verify that, given an input (given by the user), some output is returned or that the system reached a certain state. We are making sure that the system is doing what is expected. Not how it's done. Therefore, they highly focus on making sure the value is delivered.
2. They speed up development. Yes, you read me correctly. Let's look at two examples: backend and frontend development.

Backend development

Imagine that you are working on implementing an API delivered through HTTP. Whether you want it or not, you are going to end up calling your API endpoints and checking that they behave like you expect. Calling your API endpoints manually requires you to use Postman, or Chrome, or any other client. With a good E2E test setup, you can:

• write your tests,
• change your code,
• and see the results of the tests as frequently as hitting the save button of your IDE.

That way, you don't have to trigger the tests manually. You code, you see the results. Imagine your screen being split in two windows: one is your IDE, one is the testing framework constant running the tests for you.

Frontend development

Testing your frontend from end to end brings even more value. Manually interacting with a user interface is slow and tedious. Being able to automate those interactions saves enormous amounts of time. You can have your testing environment set up in a way that saving files automatically triggers the tests you pre-selected and verifies that they do what is required. You can also use those tests to reach a certain state of your application and work from there.

The tests you are writing to automate interactions can then be used and should be meant to last. They become part of the requirements that must be verified. You achieved two goals: your software is tested and an excellent developer experience (DevX). You save time and deliver quality. It's all about the value.

Common criticisms

1. E2E tests are slow: E2E tests, especially for frontend, are slower to run that integration or unit tests. That is true but also, what do you care? On a developer's machine, one can always select the tests they want to run. As part of a continuous integration pipeline, a few minutes more won't make a huge difference on your budget. It will take more time to complete until delivery but the value they bring seems to completely outweigh those longer jobs. Furthermore, in a Continuous Delivery setup, releasing software to production regularly is better done when we have the guarantee that the software is doing what it is supposed to do. And there is nothing better than E2E tests to achieve that. You can then release your software more often, as it requires less manual testing.
2. Writing tests takes time: as most skills, you get better at it by practicing. The first time you write a test, you must learn the testing framework, formulate exactly what you want to test, put it in code. And like many skills, you will be slow at the beginning. The good news is that you won't get any slower! You will get better at it pretty fast. Invest in yourself, learn this skill and save your future self from ever having to test things manually. Furthermore, we now have access to AI based code generation tools that can write the tests for you in a matter of seconds.
3. Maintaining tests take time: yes, just like any code you write. The advantage of E2E tests is that they should resist changes pretty well, assuming that the requirements of the system don't change. E2E tests are very useful to guarantee that requirements are fulfilled. Therefore, E2E tests most likely break due to a change of requirements. When it happens, they are a reminder that they must be adjusted (replaced or modified) as well.

Thanks for reading this post. I hope you are convinced now that writing E2E tests should be part of your workflow. I was skeptical at first (cf. https://blog.merckx.fr/integration-testing-setup-with-nestjs-and-mongodb/). Writing E2E tests take practice. Once it's there, you will never want to go back.