Dive Deep CPU & I/O Incentive Operations In Node.JS Event Loop

What’s Node.JS Event Loop Do?

As mentioned In Official Node.JS document: The Node.js Event Loop, Timers, and process.nextTick().

The event loop is what allows Node.js to perform non-blocking I/O operations — despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible.

We are be told that when event loop perform non-blocking I/O operations, it’s possibility offload I/O operations to libuv thread pool or kernel.

What’s libuv Worker Thread Pool Do?

As mentioned in official Node.JS document: Don’t Block the Event Loop. libuv worker thread is responsibility for

DNS: dns.lookup(), dns.lookupService().

File System: All file system APIs except fs.FSWatcher() and those that are explicitly synchronous use libuv's threadpool.

Crypto: crypto.pbkdf2(), crypto.scrypt(), crypto.randomBytes(), crypto.randomFill(), crypto.generateKeyPair().

Zlib: All zlib APIs except those that are explicitly synchronous use libuv’s threadpool.

We are be told that worker thread is just responsibility for dns, fs, crypto and zlib modules. dns.lookup() is complex operation include file and network I/O. We can classify that libuv use thread to perform file I/O and offload network I/O to kernel (using epoll). We have followed use cases to dive deep into Node.JS Event Loop handle I/O operations.

crypto.pbkdf2

Let us observe crypto.pbkdf2() when use UV_THREADPOOL_SIZE=1 .

process.env.UV_THREADPOOL_SIZE = 1;
const crypto = require("crypto");
const start = Date.now();
crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
    console.log('[worker thread][crypto.pbkdf2]', Date.now() - start, "ms");
});
crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
    console.log('[worker thread][crypto.pbkdf2]', Date.now() - start, "ms");
});
console.log("[event loop thread] done")

result:

[event loop thread] done
[worker thread][crypto.pbkdf2] 555 ms
[worker thread][crypto.pbkdf2] 1104 ms

we are be told that crypto.pbkdf2() is executed by worker thread and blocking. The crypto.pbkdf2()process is

event loop thread -> worker thread (blocking until finished) -> poll queue

fs.readFile

Let us observe fs.readFile() when use UV_THREADPOOL_SIZE=1. I put empty in /tmp/test. fs.readFile() is finished 1 ms.

process.env.UV_THREADPOOL_SIZE = 1;
const fs = require("fs")
const crypto = require("crypto");
const start = Date.now();
crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
        console.log('[worker thread][crypto.pbkdf2]', Date.now() - start, "ms");
});
fs.readFile("/tmp/test", () => {
        console.log('[worker thread][fs.readFile]', Date.now() - start, "ms");
});
console.log("[event loop thread] done")

result:

[event loop thread] done
[worker thread][crypto.pbkdf2] 558 ms
[worker thread][fs.readFile] 559 ms

We are observe worker thread will execute crypto.pbkdf2() and fs.readFile() by order.

When we change the order of two executions.

process.env.UV_THREADPOOL_SIZE = 1;
const fs = require("fs")
const crypto = require("crypto");
const start = Date.now();
fs.readFile("/tmp/test", () => {
        console.log('[worker thread][fs.readFile]', Date.now() - start, "ms");
});
crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
        console.log('[worker thread][crypto.pbkdf2]', Date.now() - start, "ms");
});
console.log("[event loop thread] done")

result:

[event loop thread] done
[worker thread][crypto.pbkdf2] 555 ms
[worker thread][fs.readFile] 556 ms

It’s surprised that fs.readFile() is slower than crypto.pbkdf2(). It’s mean worker thread offload file I/O to kernel and then released to do crypto.pbkdf2(). Kernel quickly finish I/O and notify worker thread. But worker thread is still blocking by crypto.pbkdf2(). Until finished crypto.pbkdf2(), worker thread is available to put callback to poll queue.

The process of fs.readFile() is

event loop thread -> worker thread (released) -> kernel -> worker thread -> poll queue

dns.lookup

dns.lookup() is a complex operation include file and network I/O operations.

Let us observe dns.lookup() when use UV_THREADPOOL_SIZE=1 .

process.env.UV_THREADPOOL_SIZE = 1;
const crypto = require("crypto");
const dns = require('dns');
const options = {
  family: 6,
  hints: dns.ADDRCONFIG | dns.V4MAPPED,
};
const start = Date.now();
dns.lookup("us.yahoo.com", options, (err, address, family) =>
  console.log("[worker thread][dns.lookup]", Date.now() - start, "ms"));
crypto.pbkdf2('a', 'b', 100000, 512, 'sha512', () => {
        console.log('[worker thread][crypto.pbkdf2]', Date.now() - start, "ms");
});
console.log("[event loop thread] done")

result

[event loop thread] done
[worker thread][dns.lookup] 51 ms
[worker thread][crypto.pbkdf2] 617 ms

It’s surprised that result is not equal with fs.readFile(). We suspect dns.lookup() will block thread pool.

Let us do currency lookups and observe again.

process.env.UV_THREADPOOL_SIZE = 1;
const crypto = require("crypto");
const dns = require('dns');
const options = {
  family: 6,
  hints: dns.ADDRCONFIG | dns.V4MAPPED,
};
const start = Date.now();
dns.lookup("us.yahoo.com", options, (err, address, family) =>
  console.log("[worker thread][dns.lookup][yahoo]", Date.now() - start, "ms"));
dns.lookup("github.com", options, (err, address, family) =>
  console.log("[worker thread][dns.lookup][github]", Date.now() - start, "ms"));
dns.lookup("google.com", options, (err, address, family) =>
  console.log("[worker thread][dns.lookup][google]", Date.now() - start, "ms"));
dns.lookup("facebook.com", options, (err, address, family) =>
  console.log("[worker thread][dns.lookup][facebook]", Date.now() - start, "ms"));
dns.lookup(domain1, options, (err, address, family) =>
  console.log("[worker thread][dns.lookup]", Date.now() - start, "ms"));
console.log("[event loop thread] done")

result:

[event loop thread] done
[worker thread][dns.lookup][yahoo] 177 ms
[worker thread][dns.lookup][github] 192 ms
[worker thread][dns.lookup][google] 194 ms
[worker thread][dns.lookup][facebook] 194 ms

we observe that dns.lookup() use the worker thread but not blocking worker thread. We are confused more and more about dns.lookup(), until I read the content from Node.JS document: The Node.js Event Loop, Timers, and process.nextTick().

Since most modern kernels are multi-threaded, they can handle multiple operations executing in the background. When one of these operations completes, the kernel tells Node.js so that the appropriate callback may be added to the poll queue to eventually be executed.

We are told that kernel is possible tell Node.JS to add callback to poll queue. It is multiplexing like epoll in linux. As possible process of dns.lookup() is worker thread pass DNS I/O operation to linux kernel, kernel finish I/O and put callback to poll queue.

The process of dns.lookup() is

event loop thread -> worker thread (released) -> kernel (epoll) -> poll queue

Fetch / HTTP Request

Fetch/HTTP Request is common utils to use backend to call 3rd party api. When we make http request to a ip:

fetch("127.0.0.1")

dns.lookup() will not be called. We can summary the process

event loop thread -> kernel (epoll) -> poll queue

When we make http request to a domain:

fetch("localhost")

dns.lookup() is called. We can summary the process

dns.lookup(): event loop thread -> worker thread (released) -> kernel (epoll) -> poll queue

http.request(ip): event loop thread -> kernel (epoll) -> poll queue

DB Query

DB Query is most used in web service. As we know db is TCP network I/O operation. We can summary the process

event loop thread -> kernel (epoll) -> poll queue

Summary

As far as we know, Node.JS I/O modules have different implement non-blocking I/O operation using worker thread or linux kernel. libuv perform File I/O using thread and offload Network I/O to kernel (epoll). We should separate I/O operations to different implementation.

Disk I/O Incentive

Worker thread is used but not blocking. However, worker thread is responsibility to put callback to poll queue.

HTTP Network I/O Incentive (Domain Assigned)

Worker thread is only used to do dns.lookup() but not blocking. Event loop thread is responsibility to offload HTTP Network I/O Operation to kernel.

HTTP Network I/O Incentive (IP Assigned)

Worker thread will not be used. Event loop thread is responsibility to offload HTTP network I/O operation to linux kernel.

TCP Network I/O Incentive

Worker thread will not be used. Event loop thread is responsibility to offload HTTP network I/O operation to linux kernel.

CPU Incentive

Only Zlib and part of Crypto utils are executed by worker thread. Therefore, we should prevent to do cpu incentive task in event loop thread.