Hi, it’s been a long time since the last time I wrote a post on my blog page. Those days were tough to me because I have a lot of things to worry about. But now I’m back with a new post. Note that this is just my opinion/point of view about this (of course I got some knowledge from friends, blogs)
1. Why single thread but not multi thread
Many languages use multi thread models like PHP, Java, C# but JS still chooses to stay with single thread event loop model because it has some advantages
- First thing to get into account here is that Javascript was born with the purpose of building a scripting language for front-end development. Its aim is to build a fast, easy to use language to fit into front-end development. Dealing with single thread is much more easier than multi threads where you have to face with problems of design architecture (since threads share same resources), deadlocks, race conditions and much more things that can make you crazy.
- Secondly, with multithreading model, for each request, you’ll create a new thread to handle it. The problem here is that the number of threads you can create depends on server’s RAM capacity. In single-thread, the thing that you have to worry about is CPU, not RAM. But with the speed of modern CPU which can handle millions of operations per second, for application which does not require many CPU intensive requests, using single thread model is best choice (We’ll talk about the reason why we need to care about the CPU intensive requests in the next sections where we discuss about how NodeJS handle a request), by not spawning new thread for each request, it will reduce the memory and resource usage
2. Understanding the flow of handling a request in NodeJS
a. Some basic definitions
Before diving into the main part of this section, let’s go through some basic definitions so that when we come to the next part, it would be easier to understand the secret behind NodeJS.
NodeJS built on top of V8 Engine and LibUV. In this article, the thing we need to care about is LivUB. It’s written in C, contains the Event Loop, Thread Pool (which provide more threads that can be used by NodeJS) and Event Queue. The number of threads in threads pool in LibUV by default is 4. So basically, NodeJS will have 1 main thread and 4 sub threads (I don’t know if they can be called as “sub-thread”, but just keep in mind that NodeJS does not go around with only 1 thread)
Now, let’s come to one of the biggest argument of all time among Javascript developers (also some other developer community). It is: Is NodeJS single-threaded?
- One of the proof given by the people believe that NodeJS is not single-thread is that: NodeJS built on top of V8 Engine and LibUV, LibUV has the thread pool which contains some other threads that NodeJS can use, so basically, NodeJS has more than 1 thread, therefore, it’s not true that NodeJS is single-threaded. In this argument, I’m on the side of people think that NodeJS is Single-Threaded. Let me explain why: Yeah, that’s true that NodeJS can take advantage of using threads provided by LibUV. But keep in mind that, the key word “single-threaded” is about the ability of executing the same piece of code more than 1 at the same time. For example, if you have a loop that log in the console a word, i.e: “Hello world”. In NodeJS, within an instance of NodeJS application, there can’t be 2 loops run at the same time simultaneously, you can test it yourself.
CPU bound, IO bound task
- I/O = input/output task such that: reading some rows from database and returns it, counts number of lines in a file,…
- CPU: Tasks which your CPU has to do the job. i.e: calculate a operation: 1+1 = 2, image processing.
- So just remember that I/O task is something that CPU can’t handle itself but must have the I/O subsystem to do it
- CPU task must be executed in the main thread of NodeJS, while I/O task can be executed in threads within the thread pool provided by LibUV. That’s the reason why NodeJS is not suitable for application with CPU intensive tasks. We’ll go into detail in the next few minutes.
Blocking I/O vs Non Blocking I/O:
- Blocking I/O: Requests involve with I/O operation will block the thread/process until it’s done
- Non blocking I/O: Request involve with I/O operation will be pushed into the queue and executed later (if it’s not ready, if it’s ready, returns it right away)
b. How NodeJS handle a request
Okay, it’s enough for the definition stuffs. Now go to the your main concern when you go here.
Here’s how NodeJS handle a request.
When a request comes, it is firstly pushed into a queue called event queue.
Event loop will pick up the first element in event queue and start to process it. There are 2 cases here:
- If the request is only involved with not blocking IO task / CPU task (do some calculation, no need to call other IO subsystem) (in the above example, it’s the job of calculating the bill of the order), event loop will push it to the call stack and NodeJS will execute it right away in the main thread, here comes the first bottleneck of NodeJS, when NodeJS is handling that task in the main thread, event loop won’t pick up any new task from event queue, all other tasks must waits for the processing task to be done which leads to the fact that the response time will be extremely slow. That’s the reason why NodeJS is not suitable for CPU intensive task, CPU task can only be done in the main thread, when the main thread is busy, it can’ t do anything else.
- If it’s a non blocking IO task, the event will push that job to another queue, tasks in that queue are waiting task to be done by threads in threads pool provided by LibUV. Please note that by default, LibUV only provides 4 additional threads to process blocking I/O tasks. That means when these 4 threads are busy handling tasks. Every 5th task comes in must wait (in the queue I mentioned above). But keep in mind that the number of threads provided by LibUV can be changed via settings (but as far as I know, 4 is enough to handle most of requests), if the number of requests are too big, that time we should consider about vertical scaling (create another instance of application) instead of increasing the number of LibUV’s threads.
- Some might wonders, how could it be possible that NodeJS application can handle thousands of requests while it only have 5 threads in total. The point here is “callback”. When a thread (A) in thread pool pick up a task, it will interact with other I/O subsystems like a database (for example). And note that, the database also provides some threads (B) itself, so thread A just comes in and say “hey, database, I need to get some data in the database, but I won’t wait for you to find and give it back to me, instead I will leave a “callback” here, call it when you’re done with your job, I will go back to pool and pick up another job”. Once the database’s done its job, it will call the callback function and send the response back to the callback queue, and when the call stack is empty, event loop will pick from callback queue to the call stack to continue to handle the request.
Below is a simple diagram visualize how NodeJS handle a request
Source: https://dattp.github.io/2020-04-10-event-loop-in-nodejs/
Follow-up questions:
You guys might have some doubts about this (like I did), so I will list some questions here, hope that they will (partially) match your concerns.
❓ Okay, I understand what you said in (1). But I don’t get it at all, so what will happens after that, it will send back the response to the call stack and then what?
→ Yeah, right, keep in mind that your function that handles the request is not usually simple, it might involves with many call to database, or any other blocking I/O task. Like this
const handleRequest = async (input: InputDTO) => {
const dataA = await dbConnection.get(A);
const dataB = await dbConnection.get(B);
// process the output which combining dataA and dataB or some other stuff associated with those datas.
}
So NodeJS will breaks your handleRequests function into some smaller task. For example, get dataA is 1 task, get dataB is another task, and finally processing the output is another task. each task is separately pushed into the event queue and handle individually, (of course it must be in the order that you write in your code).
- A small note in this example is that, whenever you call the “await” keyword in an async function, what comes after the await keyword will be automatically wrapped in a Promise. and NodeJS will stop execute your function at the point it meets await keyword, it will continue to process that function when it receives the response from that Promise.
When dataA is returned from the callback, NodeJS will continue to get the dataB from the database, event loop will pick it (if it’s on top of the call stack) and process it just like how it processed to get the dataA. And it will continue that loop of process until it reaches the end of handle request function.
❓ What is the priority in case call stack is empty and we have task from event queue and callback queue are ready at the same time.
- In my opinion, I think it will take the task from callback queue first, since that task from callback queue has something to do with a previous request, it’s better to send response to previous request instead of processing the new one if both of them are ready. But it’s just my assumption. Maybe NodeJS does not follow that rule.
❓ I think that callback queue contains some smaller type of queue, doesn’t it
- You’re right, when I say callback queue, I just want to talk about the high level of processing task routine in NodeJS. Indeed, we have more than 1 callback queue. They are: 1) NextTick queue 2) Micro task queue 3) Timers Queue 4) IO callback queue (Requests, File ops, db ops) 5) IO Poll queue 6) Check Phase queue or SetImmediate queue 7) close handlers queue. Event loop will continuously check these callback queues in the order and if any task in that queue is done, it will move that back to call stack to continue process (of course in case the call stack is empty).
3. Summary
Okay, above is all the thing that I want to share with you guys today. Note that all of them are based on my understanding and some articles I read on the internet, they might not be true 100% (or maybe completely wrong. If there is something wrong with this article. Please feel free to drop a comment and I will go to discuss more with you guys. Thank you and bye bye
References:
https://viblo.asia/p/ben-trong-nodejs-2-thu-vien-libuv-RQqKLRMOl7z
https://dattp.github.io/2020-04-10-event-loop-in-nodejs/
https://www.digitalocean.com/community/tutorials/node-js-architecture-single-threaded-event-loop
Top comments (0)