Is NodeJS Test Runner “Dev-Ready”?

As a dedicated advocate of testing, I am constantly on the lookout for innovative approaches and techniques to enhance the confidence in my coding practices. Whenever a new testing solution emerges, I eagerly delve into it, eager to assess its potential benefits. This insatiable curiosity led me to explore the relatively fresh offering from NodeJS itself: its own test runner.

NodeJS now offers its own test runner, which holds great promise for developers seeking reliable testing frameworks. Intrigued by its potential, I embarked on a journey to examine whether this new tool is truly ready for production and can be relied upon.

Establishing the Criteria for "Dev-Readiness"

To determine whether the new NodeJS test runner is truly "dev-ready," I began by defining my own set of criteria. These criteria can be summarized in five key aspects:

• Test Execution: How does the test suite run? Are there any structural differences compared to existing frameworks?
• Feedback Mechanism: How effectively does the runner provide feedback on test outcomes?
• Watcher Functionality: In order to facilitate Test-Driven Development (TDD) practices, or simply to monitor relevant file changes, does the runner include a reliable file watcher?
• Assertion Capabilities: How well does the runner handle assertions? Are there built-in features for robust assertion testing?
• Code Coverage: When working with unfamiliar code, I often rely on test coverage reports to gauge the level of confidence I can have when making modifications. Can the Node runner generate a comprehensive coverage report?

By thoroughly examining these five aspects, I aim to gain the necessary confidence to integrate this new test runner into my projects.

Ready to put NodeJS to the test and see how well it fulfills these criteria? Let's dive in and find out!

Hey! for more content like the one you're about to read check out @mattibarzeev on Twitter 🍻

Code mentioned in this post can be found in my node-testing-lab GitHub repo.

The NodeJS test runner reached stability with version 20, although it had been available even prior to that.
Now, let's begin from the beginning, shall we? The initial step involves creating a brand new empty project, configuring yarn within it, and ensuring that NodeJS version 20 is being utilized. To accomplish this, I use nvm, a tool that enables the installation and management of different NodeJS versions. Here's how I go about it:

nvm install v20.0.0

And make sure I use it:

nvm use v20.0.0  

Cool, we’re good to go.

Test Execution

I will create a simple “Add” function using TDD methodology and let’s see how it goes -
The service I’m planning on creating resides on a file called calc-service.js and I have another file for the test called calc-service.test.js, both are empty at this stage.

Note: we're not using babel at this stage, so in order for node to be able to work with ESM imports we should declare the project’s type as “module”. This is done in the package.json file

For now let’s start with a simple check to make sure that my runner is actually working. No assertions yet:

import {describe, it} from 'node:test';


describe('calc-service', () => {
   it('should work...', () => {});
});

Since I would like to keep the same testing paradigm I’m familiar with, I am importing “describe” and “it” to resemble Jest testing.
For running the test I simple type this command in the terminal:

node --test

And I get this output, which is kinda ugly but at least we can see where we at:

TAP version 13
# Subtest: /home/matti/my/labs/node-testing-react/src/services/calc-service.test.js
ok 1 - /home/matti/my/labs/node-testing-react/src/services/calc-service.test.js
  ---
  duration_ms: 59.507962
  ...
1..1
# tests 1
# pass 1
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 63.401323

Criteria: How does the test suite run? Are there any structural differences compared to existing frameworks?
Answer: The test suite runs with a simple terminal command and there are no structural differences from Jest or Mocha tests.

Feedback Mechanism

Since the TAP report format is difficult to comprehend, let's switch to a more visually appealing and readily available option called the "spec" reporter. To achieve this, I include a script in my package.json file that specifies the desired reporter. From now on, I will use the following command in my terminal to run the tests:

"scripts": {
       "test": "node --test --test-reporter spec"
   }

And the outcome looks like this:

Ahh… much better. I have colors and I can see that my test passes with an indication of how long it took. Swell.

Criteria: How effectively does the runner provide feedback on test outcomes?
Answer: It’s good enough, we see the details we need and if the test fails we get an indication about the relevant lines. Perhaps too verbose, but that’s ok.

Watcher Functionality

Prior to making any changes to the code we’ve just created, it is essential to have continuous feedback from the tests. To achieve this, we require a watch mechanism that monitors the test file and all its associated files, triggering a test run whenever any of these files are modified.

To implement this functionality, I have created an npm script that watches all the test files within the src directory. Here is the script I have developed:

"test:watch": "node --test --test-reporter spec --watch ./src/**/*.test.js"

Note: the watch param has to have a file path value to watch on, otherwise it will not work.

Criteria: In order to facilitate Test-Driven Development (TDD) practices, or simply to monitor relevant file changes, does the runner include a reliable file watcher?
Answer: Yeah :)

Ok, now that we have ongoing feedback we can start with some assertions.

Assertion Capabilities

I start with testing the “add” method, adding 1 and 2, expecting the result to be 3. Here is the test:

import {describe, it} from 'node:test';
import assert from 'node:assert';
import {add} from './calc-service.js';


describe('calc-service', () => {
   it('should return a result of adding 2 numbers', () => {
       const result = add(1, 2);
       assert.equal(result, 3);
   });
});

The assertion is imported from the node:assert module, which offers various types of assertions.
As expected, the initial test fails immediately, prompting us to address the issue. However, in this instance, I won't strictly adhere to Test-Driven Development (TDD) principles by implementing only the minimum required to pass the test (If you’re interested in that, you can explore other posts I have written, such as "Creating a React Component with TDD" or "Creating a React Custom Hook using TDD" among others)

The test passes, blazingly fast, if I may add.

Criteria: How well does the runner handle assertions? Are there built-in features for robust assertion testing?
Answer: Sure looks like it. There are many assertions types which seem to support advanced test use-cases.

Code Coverage

What makes code coverage significant? Well, it's not merely about impressing your manager with a high percentage. Code coverage holds importance because it provides crucial insights into the safety of code refactoring. While it doesn't guarantee absolute certainty, having extensive code coverage brings a greater sense of confidence when making modifications to your code. This is precisely why it is imperative to be able to visualize and analyze your code coverage.

Jest provides a convenient coverage solution by utilizing Istanbul/nyc to generate comprehensive reports that include line-by-line code coverage details and highlight any areas lacking coverage. Similarly, I aspire to have a similar functionality for the NodeJS test runner.

Interestingly, NodeJS itself offers built-in coverage support. By defining the NODE_V8_COVERAGE environment variable and specifying a target directory, NodeJS generates V8 coverage data. However, this is merely the initial step, as we aim to transform this data into a more visually appealing format that can be easily understood by humans.

Fortunately, there exists a helpful package called "c8" that simplifies this entire process for us. It takes care of setting the NODE_V8_COVERAGE environment variable and converts the coverage data, generating a comprehensive text report directly in your terminal.

To incorporate the "c8" package into the project, I install it, and then add the following script to the package.json file:

"test:coverage": "c8 node --test"

And after running it I’m getting this in the terminal:

That’s really nice, but I would like a cool HTML coverage report with all the details.

Once again, "c8" proves to be reliable and allows us to set the reporter to "html." While it is possible to specify the reporter directly in the npm script, I prefer creating a separate configuration file for "c8" as it provides easier maintenance in the long run.

Introducing the .c8rc.json configuration file, which now includes two reporters (to satisfy my desire for both the text report on the command line and the HTML report). Here is the configuration file:

{
   "reporter": ["text", "html"]
}

Here is the nice HTML result we get, just like we are used to:

Criteria: Can the Node runner generate a comprehensive coverage report?
Answer: Yep!

Wrapping up

In conclusion, it is evident that NodeJS surpasses expectations by providing a robust test runner that can handle basic testing tasks without relying on external frameworks like Jest or Mocha. Emphasizing the utilization of native features over third-party dependencies yields better outcomes.
Is it “dev-ready”? I would have to say that “Yes, I believe it is”.

However, there are still aspects left unexplored. One such aspect is mocking, which appears to have a promising approach according to the documentation but requires further investigation.
Additionally, how can we tackle testing code that requires transpilation, such as React? While Jest offers transformers for this purpose, the question remains whether similar capabilities can be integrated into the NodeJS ecosystem.

These unanswered questions provide food for thought, encouraging us to delve deeper into the possibilities that NodeJS presents. Until then, let's continue our exploration and keep pushing the boundaries of testing within this dynamic environment. Stay tuned for more insights in future posts :)

Code mentioned in this post can be found in my node-testing-lab GitHub repo.

Hey! for more content like the one you've just read check out @mattibarzeev on Twitter 🍻

Photo by Nathan Dumlao on Unsplash