Skip to content

Worker threads vs cluster vs horizontal scaling

Processes share only the port, threads share only what you hand them, and the libuv pool is one fixed crew per process.

Predict first

Before you read: predict

ExamHall's login handler keeps locked candidates in a module-level Set and rejects every login for a candidate in it. The API now runs under node:cluster with 4 workers. An invigilator locks one candidate; that single request adds the candidate to the Set. A bot then sends 12 login attempts for that candidate at once, each on a new connection.

Which of the 12 attempts are rejected?

Pick one

The 30-second version

A Node process runs its JavaScript on one thread. Cluster or more containers add processes: each has its own memory, and each connection stays on one of them. Worker threads run JavaScript in parallel inside one process; messages are copies, and only a SharedArrayBuffer is shared. fs, dns.lookup and async crypto share one 4-thread pool per process.

Processes share only the port, threads share only what you hand them, and the libuv pool is one fixed crew per process.

How it works

Step 1 of 13

One thread per process. A Node process runs all its JavaScript on one main thread; the prerequisite shows what one long callback does to it.
In the kitchen
  • the front-desk clerk, who serves every walk-in in turn → the main thread (step 1)

Analogy: A translation agency with branch offices

One agency, one front-desk clerk per office, and some shared help.

In the agency In Node
the front-desk clerk, who serves every walk-in in turn the main thread (step 1)
branch offices in other towns, each with its own client files processes: cluster workers or containers (steps 2, 5)
head office's switchboard puts each new caller through to a branch; a caller who stays on the line stays with that branch the primary deals connections; a kept-alive one stays put (steps 3–4)
a back-office translator in the same building, handed a photocopy of each document a worker thread and postMessage (steps 6–7)
the whiteboard on the wall, which the clerk and the translator both read a SharedArrayBuffer (step 8)
four archive clerks who fetch files from the basement for everyone in the building; how many is decided before the office opens the libuv pool and UV_THREADPOOL_SIZE (steps 10–12)

In real terms: before scaling, ask where each piece of state lives: in one process, in one thread, or in a store every process reads.

Breaks down when: the translator's notes on a photocopy could be posted back and filed. A worker's changes reach the main thread only as a new message, itself a copy (step 7).

Contrasting case: a Java or Go server runs its handlers on many threads that share one heap, so shared maps need locks. Node processes share nothing, so they need a store instead.

Examples, run for real

4.1 ExamHall: a flag that only some dashboards saw

EXH-1340: the proctor API moved to 4 cluster workers. A proctor's flag on a candidate went into a module-level Set in whichever worker took that request. Dashboards poll on new connections, so most polls reached other workers. MR !1540 keeps flags in Redis, which every worker reads. Locally, the script stands in for Redis with a store in the primary that workers query over IPC.

4.1 as shipped, flags in each worker's memory:      dashboard reads that saw the flag 2 of 8
4.1 MR !1540, flags in one shared store (stand-in): dashboard reads that saw the flag 8 of 8

Only the polls dealt back to the flagging worker saw the flag (steps 3, 5). Nothing failed; the workers simply disagreed. Sessions, rate limits and websocket rooms drift apart the same way.

4.2 Four threads, one counter

Four worker threads each add 1 to a counter in one SharedArrayBuffer, 1000 times, with Atomics.add.

4.2 4 worker threads × 1000 Atomics.add: counter on the main thread 4000 · distinct thread ids 4 · all in the main thread's process: true

The threads are separate (four thread ids) but live in one process, and the shared buffer is the one place all of them write (steps 6, 8). With Atomics, no increment is lost.

4.3 A full pool, and a server that still answers

Four file reads are stuck, each holding one pool thread. Then the process starts an fs.stat and receives an HTTP request.

4.3 default pool, 4 threads held by stuck reads: fs.stat done before release false · HTTP request answered true

The fs.stat waits for a free pool thread; the request, on a socket, never needed one (steps 10–11). A slow disk can stall unrelated file work while the API still looks healthy.

On Windows, cluster's default policy leaves distribution to the OS. In one ungraded measurement (sched-split.mjs), one worker took all 12 connections in every one of 50 runs; with round-robin forced, each took 3.

Picture it

on the left, a grey-haired man in a green shirt and apron chisels a board at a large wooden workbench, with a pegboard of hand tools (a square, pliers, a hammer, chisels, planes) behind him. In the middle, a man in a blue jacket sits at a separate white desk under a magnifier lamp, working on small gears with tweezers. On the right, drawn smaller and further away, a woman shapes a pot on a potter's wheel and a man hammers on an anvil. There is no text and no wall between the three areas.
An illustration to help it stick. The mechanism is in the sections above.

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

A catalog API renders product thumbnails in JavaScript, about a second of CPU each. While four thumbnails are rendering, even /health goes unanswered until one finishes. The container has 4 cores.

Which change keeps /health answered while four thumbnails render?

Pick one

Question 2 of 5

A payouts service runs its statement exports in a worker thread. Four exports are stuck there, each in an fs.readFile from a hung network drive. Meanwhile, on the main thread, the load balancer's /health request arrives, and the service calls crypto.randomBytes(16, cb) for a payout token and dns.lookup('localhost', cb). The thread pool has its default size.

Which of the three complete before the drive recovers?

Pick one

Question 3 of 5

A streaming API runs under node:cluster with 4 workers. A load test in one Node process sends 40 requests, one after another, with http.get and Node 24's default agent. The metrics show one worker handled all 40 and the other three none.

Which is the bug?

Pick one

Question 4 of 5

A ticketing service reads ticket templates with fs.readFile and makes ticket codes with crypto.randomBytes(16, cb). When four template reads hang on a slow network drive, the service stops issuing tickets until the drive recovers.

Which one change keeps other template reads and ticket codes flowing while up to four reads hang?

Pick one

Question 5 of 5

A chat server's main thread keeps flagged-message counts per room in an Int32Array, counts, and hands it to a moderation worker with worker.postMessage(counts). The worker flags 5 messages in room 1, adding 1 to counts[1] each time. The dashboard, reading counts on the main thread, shows 0 for every room.

Which is the bug?

Pick one

Go deeper

Pro tips
  1. Scale out only stateless handlers. Before adding cluster workers or replicas, grep for module-level Map, Set and counters that requests write, and move them to Redis. When: any change from one process to several (step 5).
  2. On Windows, force round-robin before you compare worker counts. Set NODE_CLUSTER_SCHED_POLICY=rr, or cluster.schedulingPolicy, before the first fork. When: local load tests on Windows (step 3). Verify (RAN, sched-split.mjs, 50 runs per line, ungraded): SCHED_NONE put all 12 connections on one worker every time; SCHED_RR gave 3/3/3/3.
Source
  • Node.js v24 API, Cluster: https://nodejs.org/docs/latest-v24.x/api/cluster.html · fetched 2026-09-28. Round-robin is "the default one on all platforms except Windows", in which "the primary process listens on a port, accepts new connections and distributes them across the workers in a round-robin fashion" (step 3). The second approach leaves accepting to the workers, and "distribution tends to be very unbalanced due to operating system scheduler vagaries" (step 3, sched-split.mjs). "The worker processes are spawned using the child_process.fork() method" and "workers are all separate processes" (step 2). "Node.js does not provide routing logic. It is therefore important to design an application such that it does not rely too heavily on in-memory data objects for things like sessions and login" (step 5). schedulingPolicy is "effectively frozen once either the first worker is spawned, or .setupPrimary() is called"; "SCHED_RR is the default on all operating systems except Windows"; it "can also be set through the NODE_CLUSTER_SCHED_POLICY environment variable" (tip 2). Valid as of 2026-09-28.
  • Node.js v24 API, HTTP, http.globalAgent: https://nodejs.org/docs/latest-v24.x/api/http.html#httpglobalagent · fetched 2026-09-28. "Global instance of Agent which is used as the default for all HTTP client requests. Diverges from a default Agent configuration by having keepAlive enabled and a timeout of 5 seconds". Valid as of 2026-09-28.
  • Node.js v24 API, Worker threads: https://nodejs.org/docs/latest-v24.x/api/worker_threads.html (source doc/api/worker_threads.md at tag v24.19.0) · fetched 2026-09-28. "Workers (threads) are useful for performing CPU-intensive JavaScript operations. They do not help much with I/O-intensive work" (step 9). "Unlike child_process or cluster, worker_threads can share memory. They do so by transferring ArrayBuffer instances or sharing SharedArrayBuffer instances" (steps 7–8). port.postMessage(): the value is transferred "in a way which is compatible with the HTML structured clone algorithm", and transferred objects "are not usable on the sending side of the channel anymore" (step 7). Inside a worker, process.env "is a copy of the parent thread's environment variables"; "Changes to one copy are not visible in other threads". The page does not state in one sentence that each worker has its own module instances; step 6's module claim is observed. Valid as of 2026-09-28.
  • Node.js v24 API, Command-line options, UV_THREADPOOL_SIZE=size (source doc/api/cli.md at tag v24.19.0): https://nodejs.org/docs/latest-v24.x/api/cli.html#uv_threadpool_sizesize · fetched 2026-09-28. APIs that use the pool: "all fs APIs, other than the file watcher APIs and those that are explicitly synchronous", "asynchronous crypto APIs such as crypto.pbkdf2(), crypto.scrypt(), crypto.randomBytes()…", "dns.lookup()", "all zlib APIs, other than those that are explicitly synchronous" (step 10). If one of them "takes a long time, other (seemingly unrelated) APIs that run in libuv's threadpool will experience degraded performance"; the default is 4; "setting this from inside the process using process.env.UV_THREADPOOL_SIZE=size is not guranteed to work as the threadpool would have been created as part of the runtime initialisation much before user code is run" (steps 11–12; the docs' spelling). Valid as of 2026-09-28.
  • libuv, Thread pool work scheduling: https://docs.libuv.org/en/v1.x/threadpool.html (source docs/src/threadpool.rst, v1.x) · fetched 2026-09-28. "Its default size is 4, but it can be changed at startup time by setting the UV_THREADPOOL_SIZE environment variable to any value (the absolute maximum is 1024)" (step 12). "The threadpool is global and shared across all event loops" (step 11: a worker thread's loop uses the same pool). Valid as of 2026-09-28.
  • Node.js v24 API, DNS, "Implementation considerations": https://nodejs.org/docs/latest-v24.x/api/dns.html#implementation-considerations · fetched 2026-09-28. dns.lookup() "is implemented as a synchronous call to getaddrinfo(3) that runs on libuv's threadpool"; dns.resolve*() "do not use libuv's threadpool" (step 10). Valid as of 2026-09-28.
  • Node.js v24 API, Crypto, crypto.randomBytes(): https://nodejs.org/docs/latest-v24.x/api/crypto.html#cryptorandombytessize-callback · fetched 2026-09-28. "This API uses libuv's threadpool"; "The asynchronous version of crypto.randomBytes() is carried out in a single threadpool request" (step 10). Valid as of 2026-09-28.

Related lessons

Checked against the tools and docs as of 2026-09-28.