libuv & the thread pool

The waiter and the cooks

Node.js runs your JavaScript on one thread. A thread is one line of work that the computer runs step by step. Yet one Node server can talk to thousands of users at once. It does this with help from libuv, a library that sits under Node. A library is code that other programs use.

libuv talks to the operating system: the main program of the computer, like Linux, macOS, or Windows. Each operating system works in its own way. libuv hides those differences, so Node works the same way on all of them.

libuv splits the work into three jobs. Keep them apart in your head:

• The event loop runs your JavaScript callbacks, one at a time. A callback is a function you hand over now, for Node to call later.
• The operating system watches network connections. It tells the event loop when one has data ready.
• The thread pool is a small group of helper threads, called workers. They do some slow jobs, like reading files, away from the event loop. When a job is done, they tell the event loop.

"Async", short for asynchronous, means a function returns before its work is finished. It does not tell you which of these three paths the work takes.

This lesson builds on The Node.js event loop, where you saw the loop's phases. Here you'll learn three things:

• which work stays on the event loop,
• which work waits for a worker in the thread pool,
• and which work the operating system tells Node about, with no worker at all.

Handles and requests

libuv uses two words for the things it keeps track of:

• A handle lives a long time and can report many events. A server is a handle: it can get a new connection now, and another one later. A timer is a handle too.
• A request is one short job that finishes once. Reading one file is a request. Once the file is read, the request is done.

1const serverHandle = "watch for connections";2const fileRequest = "read one file";3 4console.log(serverHandle);5console.log(fileRequest);

Lines 1 and 2 save two plain strings. They are labels for the idea, not real libuv code. Line 1 stands for a server, a handle that watches for connections. Line 2 stands for one file read, a request. Lines 4 and 5 print watch for connections and read one file.

Handles and requests often work together. A network connection is a handle: Node watches it for as long as it stays open. Sending one reply on it is a request: it is done once the reply is sent. To tell them apart, ask one question: does it keep watching, or does it finish once?

The table below compares the two. Read each row as: this word means this, and here is an example from Node.

How Node hears about the network

A socket is one network connection, like the link between your server and one user's browser. Data on a socket arrives whenever the other side sends it. Node does not keep a worker waiting at each socket. Instead, libuv gives the operating system a list of sockets to watch.

The operating system tells libuv when a socket is ready. Ready means one of these:

• it has data to read,
• it can take more data to send,
• or it has closed.

This message is called a readiness notification. Because of it, one Node server can keep thousands of connections open with no extra workers.

1server.on("connection", (socket) => {2  socket.on("data", (chunk) => console.log(chunk.length));3});

Line 1 hands Node a callback for each new connection, then moves on at once. Line 2 hands over another callback, for data that arrives later on that connection. When data arrives, line 2 prints how many bytes came in. This code needs a real Node server, so you can read it here but not run it.

Each operating system has its own tool for watching sockets:

• Linux uses epoll.
• macOS uses kqueue.
• Windows uses IOCP.

libuv hides these differences, so your callbacks look the same everywhere. The reading and writing on sockets happens on the event loop's own thread. It does not go to the thread pool.

The thread pool

Files work differently from sockets. Not every operating system can tell libuv "this file is ready" in the same way. So libuv reads and writes files in its thread pool: a small group of worker threads that the whole program shares. Each worker takes one job, does it, and then takes the next.

The pool also does a few other slow jobs, like looking up a website name. When a worker finishes a job, libuv tells the event loop. Then the event loop runs your callback.

1const jobs = [2  { name: "menu.json", duration: 3 },3  { name: "orders.json", duration: 2 },4  { name: "users.json", duration: 4 },5  { name: "prices.json", duration: 1 },6  { name: "notes.json", duration: 2 },7  { name: "report.json", duration: 3 },8];9 10console.log(schedulePoolJobs(2, jobs));

Lines 1 to 8 list six file jobs. Each has a made-up length of time, called duration. Line 10 calls schedulePoolJobs, a small helper written for this lesson. It shares the jobs between two workers. It is not real libuv code, and it does not time your disk. It shows one rule: when every worker is busy, the next job waits.

The helper gives each job to the first worker that is free. The picture shows the result for two workers.

1const jobs = [Run2  { name: "menu.json", duration: 3 },3  { name: "orders.json", duration: 2 },4  { name: "users.json", duration: 4 },5  { name: "prices.json", duration: 1 },6  { name: "notes.json", duration: 2 },7  { name: "report.json", duration: 3 },8];9 10console.log(schedulePoolJobs(2, jobs));

Press Step to walk through the replay. Each step shows the jobs that end at one time. The replay runs this lesson's model. It doesn't look inside real libuv workers.

By default, libuv's pool has 4 workers. You can change the number with UV_THREADPOOL_SIZE. That is an environment variable: a setting you give a program when you start it. Before you change it, keep in mind:

• The pool is shared by the whole Node program.
• More workers use more memory.
• More workers don't help a slow callback of your own, because callbacks don't run in the pool.

So measure first, before you change it.

UV_THREADPOOL_SIZE=8 node server.mjs

This command sets UV_THREADPOOL_SIZE to 8, then starts server.mjs with Node. Set it in the command that starts Node, as here. Once the pool has started, changing the setting does not resize it. More workers can also fight over the same CPU cores and make things slower. Treat the number as a setting you measure, not a quick speed fix.

In the playground, change the number of workers. Watch when each job ends, and when the last one ends.

1const jobs = [2  { name: "menu.json", duration: 3 },3  { name: "orders.json", duration: 2 },4  { name: "users.json", duration: 4 },5  { name: "prices.json", duration: 1 },6  { name: "notes.json", duration: 2 },7  { name: "report.json", duration: 3 },8];9 10console.log(schedulePoolJobs(2, jobs));

Which Node functions use the pool

An async function returns before its work is done. That does not tell you how the work gets done. The function might wait on the operating system, or it might wait for a worker in the pool.

This matters when many pool jobs run at once. A slow batch of file reads can make an unrelated crypto job wait too, because they share the same workers.

1import { readFile } from "node:fs";2readFile("menu.json", "utf8", (error, text) => console.log(error ?? text));3console.log("request submitted");

Line 1 imports readFile from Node's fs (file system) module. Line 2 asks for one file read and hands over a callback. That is one request for the pool. Line 3 prints request submitted first. The callback runs later, once the read is done. It prints the file text, or the error if the file is missing. This code needs Node and a real file, so you can read it here but not run it.

The table shows where common kinds of Node work go. Read each row as: this work is handled here, and this is why.

Node's docs list these users of the pool:

• Most async fs functions, which read and write files.
• dns.lookup, which turns a website name into a number address.
• Async zlib functions, which make data smaller (compress it) or unpack it again.
• Some async crypto functions, which do security math: pbkdf2, scrypt, and async randomBytes.

Network sockets do not use the pool. The operating system tells Node when they are ready.

import { pbkdf2 } from "node:crypto";

const jobs = Array.from({ length: 5 }, (_, index) => new Promise((resolve, reject) => {
  pbkdf2("password", "salt", 1_000, 16, "sha256", (error) => {
    if (error) reject(error);
    else resolve(index + 1);
  });
}));

console.log("submitted", jobs.length);
await Promise.all(jobs);
console.log("all finished");

Line 1 imports pbkdf2, a slow function that turns a password into a secret key. Lines 3 to 8 start five pbkdf2 jobs and wrap each one in a promise. Line 10 prints submitted 5. Line 11 waits for all five jobs. Line 12 prints all finished.

The pool has four workers by default, so at least one job must wait for a free worker. Which job ends first, and how long each one takes, can change from run to run. Don't write code that depends on them.

JavaScript can still block

The thread pool does not run your JavaScript callbacks. Once Node calls a callback, that callback runs to the end without a break. While it runs, the event loop can do nothing else. We say the callback blocks the event loop.

A long loop, a huge piece of JSON to read, or a slow regular expression can all block it. Then timers, new network data, and every other callback have to wait.

1setTimeout(() => console.log("timer"), 0);2const end = Date.now() + 30;3while (Date.now() < end) {}4console.log("busy work done");

Line 1 sets a timer with a delay of 0 milliseconds. Line 2 picks an end time, 30 milliseconds from now. Line 3 loops and does nothing until that time comes, which keeps JavaScript busy. Line 4 prints busy work done. Only after the whole script ends can the timer's callback run and print timer. This code works the same way in a web page.

The lesson only promises the order, not exact times: the busy code always finishes before the timer callback runs. On a server, this is a real danger. Say the time a callback takes grows with the size of its input. Then an attacker can send a huge input on purpose, and the server stops answering everyone else. This is called a denial of service (DoS) attack. It can happen even though the rest of the server uses async code.

Slow patterns and huge JSON

A regular expression, or regex, is a pattern for matching text. For example, /^a+$/ matches text made only of one or more letter as. Some patterns take a very long time on certain inputs. An attacker can use that to block the server. This is called ReDoS, short for regular expression denial of service.

A warning sign is a repeat inside a repeat, like (a+)+. Take the pattern /^(a+)+$/ and a string of as that ends in !. The match must fail because of the !. But before it gives up, the regex tries every way to split the as into groups. Each extra a can double the work.

1const input = "aaaa!";2console.log(/^(a+)+$/.test(input));

Line 1 uses a short input, so it finishes quickly here. Line 2 prints false, because the ! at the end doesn't fit the pattern. The danger is not this short input. The danger is taking a long input from a stranger and expecting the match to be quick.

Huge JSON is a simpler problem. JSON.parse turns JSON text into an object, and JSON.stringify turns an object into JSON text. Both run from start to finish without a break. Their time grows in step with the size of the text. Still, a huge text can block the event loop for a long time. This is called JSON DoS. To protect against it:

• Limit the size of a request body before you parse it.
• Keep the JSON shapes you accept simple.
• Move truly slow work out of the callback.

1const body = '{"user":"Asha"}';2console.log(body.length <= 100 ? JSON.parse(body).user : "too large");

Line 1 is a short JSON text. Line 2 checks that it is at most 100 characters long before it parses it. The text is short enough, so line 2 prints Asha. A real server should check the size while it reads the request. Checking after a huge body is already stored in memory is too late.

Split long work, or move it to a worker

Some work is simple but long, like adding up a long list of numbers. You can split it into steps:

1. Do a small piece of the work, called a chunk.
2. Remember how far you got.
3. Give the event loop a turn before the next chunk.

This is called partitioning. Between chunks, the event loop can run other waiting callbacks. Splitting doesn't make the work faster, and it doesn't use a second CPU core.

1function sumInChunks(numbers, done) {2  let index = 0;3  let total = 0;4  function nextChunk() {5    const stop = Math.min(index + 2, numbers.length);6    while (index < stop) total += numbers[index++];7    if (index < numbers.length) return setImmediate(nextChunk);8    done(total);9  }10  nextChunk();11}12 13setTimeout(() => console.log("timer"), 0);14sumInChunks([1, 2, 3, 4], (total) => console.log("sum", total));

Line 1 defines sumInChunks. Line 5 picks where this chunk stops: two numbers on, or the end of the list. Line 6 adds those numbers to the total. Line 7 checks if any numbers are left. If so, it calls setImmediate(nextChunk). setImmediate asks Node to run a function soon, after other waiting work gets a turn. When no numbers are left, line 8 hands the total to the done callback.

Line 13 sets a timer, and line 14 starts the sum. The timer can run between the two chunks. It may print before or after sum 10, depending on timing. What matters is that it gets a chance between chunks. This code needs Node, so you can read it here but not run it.

For heavy CPU work, use worker_threads. This Node module starts a worker thread: a second thread with its own JavaScript, which can run on another CPU core. Moving work there is called offloading. It has a cost:

• The data you send is copied or moved to the worker.
• The result comes back to the event loop as a message, and the event loop still sends the reply.

But the main callback no longer does the whole calculation, so other users don't wait for it.

1import { Worker } from "node:worker_threads";2 3const worker = new Worker(new URL("./price-worker.mjs", import.meta.url));4worker.postMessage({ prices: [3, 4, 5] });5worker.once("message", (total) => console.log(total));

Line 1 imports Worker from Node's worker module. Line 3 starts one worker, which runs the file price-worker.mjs. Line 4 sends it a list of prices. Line 5 waits for one message back, the total, and prints it.

Don't use the libuv thread pool for your own JavaScript work. It is meant for Node's built-in file, crypto, and other jobs. Use worker threads for your own heavy work.

Good habits for a Node server

Start with short callbacks and limits on input size. Async file, crypto, website-name, and compression jobs share the same pool, so they can wait in line behind each other. A slow pool job leaves fewer workers for the next job. A slow JavaScript callback is worse. It blocks every callback on the event loop, even ones whose data has already arrived.

• Use the async file and crypto functions when you handle requests. Avoid the Sync versions, like readFileSync.
• Put limits on request bodies, regex input, the size of each job, and how big caches can grow.
• Measure how long jobs wait, and how busy the CPU is, before you change UV_THREADPOOL_SIZE.
• Split simple loops into chunks. Use worker_threads for long, heavy CPU work.
• Keep socket callbacks short, even after the operating system says a socket is ready.

This is why Node is good for servers that mostly wait for files and the network. One event loop can look after many waiting sockets. But your code has to be fair. If a request handler does too much work for one user, every other user waits for the same event loop or pool.

Common misconceptions

• “Async means a new thread.” Many network jobs use no extra thread at all. The operating system tells the event loop when a socket is ready.
• “The pool runs my callback.” A pool worker only does the job, like reading the file. Your callback then runs on the event loop.
• “More pool workers fix slow JavaScript.” A busy while loop, regex, or JSON.parse still blocks the event loop. Pool workers don't run it.
• “A timer of zero runs right away.” It runs only after the current JavaScript finishes and the event loop gets a turn.
• “ReDoS needs a huge regex.” A short pattern with a repeat inside a repeat can be enough, if the input is long and has no size limit.

The table compares words that sound alike. Read each row as: this word means this, but not that.

Practice exercises

setTimeout(() => console.log("timer"), 0);
const end = Date.now() + 30;
while (Date.now() < end) {}
console.log("busy work done");

const handle = "timer";
const request = "one file read";
console.log(handle, request);

console.log(schedulePoolJobs(1, [{ name: "a", duration: 2 }, { name: "b", duration: 1 }]).map((job) => job.finish).join(","));

const input = "aaaa!";
console.log(/^a+!$/.test(input));

Check your understanding

For each question, first name the path: a callback on the event loop, a ready socket from the operating system, or a job in the thread pool.

Key takeaways

• A handle lives a long time and can report many events. A request is one short job that finishes once.
• The operating system tells Node when a network socket is ready, with tools like epoll, kqueue, and IOCP.
• libuv's shared thread pool has four workers by default. It does file, website-name lookup, crypto, and compression jobs.
• A long loop, a slow regex, or a huge JSON.parse still blocks the event loop.
• Split simple work into chunks. Move long, heavy CPU work to worker threads.

Coming next: Browser architecture, where you'll see how a browser splits its work into separate programs, called processes, and threads.