Parallel operations in Node.js (Thread Pool)

Recently I had an Interview where I got the question "How many async operations can node js handle simultaneously?" the interviewer expected to hear 4, however, my answer was that for me it is 12 cause I have 12 logic threads in my CPU. My answer wasn't welcomed so let's do a little experiment to prove that my answer was correct!

We begin with a simple hash operation that can be performed by using the inbuilt node:crypto module, hashing is a perfect fit since it requires some CPU capacity. For time measurements I will use a difference of dates, it is not the best option but simple enough to see the correlation.

import crypto from 'node:crypto';

const start = Date.now();

crypto.pbkdf2Sync("password", "salt", 100000, 512, 'sha512')
console.log('Hash: ', Date.now() - start)

We see that one sync hash operation takes about 319ms, from here we can guess that the second log will be at about 640ms, the third 960ms etc. And if we change the code we get the result that proves it:

import crypto from 'node:crypto';

const start = Date.now();

crypto.pbkdf2Sync("password", "salt", 100000, 512, 'sha512')
console.log('Hash: ', Date.now() - start)
crypto.pbkdf2Sync("password", "salt", 100000, 512, 'sha512')
console.log('Hash: ', Date.now() - start)
crypto.pbkdf2Sync("password", "salt", 100000, 512, 'sha512')
console.log('Hash: ', Date.now() - start)

This simple diagram parents how this looks in graph format:

Ok, now that we have an understanding of sync behaviour let's switch to the async and measure its limits. As we know JavaScript is a single-threaded language, however NodeJS has not only the EventLoop but also a thread pool that is handled by libuv. So the question how many threads are available to handle the required operations?

I changed our example slightly to have more control over hashing calls amount and used pbkdf2 which is async and accepts a callback.

import crypto from "node:crypto";

const CALLS_AMOUNT = 4;
const start = Date.now();

for (let i = 0; i < CALLS_AMOUNT; i++) {
  crypto.pbkdf2("password", "salt", 100000, 512, "sha512", () => {
    console.log(`Hash ${i+1}: `, Date.now() - start);
  });
}

As we can see the difference in time is not that big so we can conclude that all operations executed simultaneously, the diagram shows this:

Let's double the number of transactions:

const CALLS_AMOUNT = 8;

As we can see first four calls are executed simultaneously then executed next four, on the diagram this looks like:

From this point, many may conclude that in NodeJS we can perform 4 asynchronous operations at the same time, but is this really the case?

Here I need to remark that the Thread Pool has size and by default this size equals 4, though we can control its size it still has limitations like logic threads of your CPU. Size can be controlled using UV_THREADPOOL_SIZE environment variable like:

import crypto from "node:crypto";

process.env.UV_THREADPOOL_SIZE = 12

const CALLS_AMOUNT = 12;
const start = Date.now();

for (let i = 0; i < CALLS_AMOUNT; i++) {
  crypto.pbkdf2("password", "salt", 100000, 512, "sha512", () => {
    console.log(`Hash ${i+1}: `, Date.now() - start);
  });
}

I set the UV_THREADPOOL_SIZE to 12 because my PC has 6 cores and 12 threads. In case I set the value over the 12, next operations will be executed it next time interval e.g. for 16 calls:

Conclusion

The in NodeJS the amount of async or I/O operations that can be performed simultaneously depends on the Thread Pool size that is limited by your CPU threads