Backpressure, admission control and priority shedding

When work arrives faster than it drains, push back early: queues sized to their callers' wait, admission by deadline, and shedding by criticality.

Before you read: predict

ExamHall's autosave service writes candidates' answer saves one at a time, 200 a second. At 09:00 a fault upstream sends it 280 saves a second for ten seconds, starting from an empty queue. Each browser waits 1 second for a save's reply, then stops waiting. The service does not check whether a browser is still waiting before it writes a save.

About how many more saves in those ten seconds does it finish while their browser is still waiting if it always takes the newest waiting save first than if it always takes the oldest waiting save first?

The 30-second version

Backpressure makes a producer wait for its consumer: a stream's write() returns false until 'drain', and a pull-based consumer asks only when free. Bound a queue at drain rate × its callers' wait, admit only what can finish in time, and shed the lowest criticality first. Under a backlog, serving newest-first finishes requests whose callers still wait.

When work arrives faster than it drains, push back early: queues sized to their callers' wait, admission by deadline, and shedding by criticality.

How it works

Step 1 of 14

Backpressure. A consumer that falls behind makes its producer wait, so the backlog stays at the source instead of growing in memory.

In the kitchen

the packhouse radios the pickers to stop when the intake bay is full, so fruit waits on the plants → backpressure upstream (steps 1–2)

Analogy: A strawberry farm's packhouse in the June harvest (horticulture)

The packing line packs a fixed number of trays an hour, and picked fruit must be chilled within four hours or it is fit only for jam.

In the packhouse	In the system
the packhouse radios the pickers to stop when the intake bay is full, so fruit waits on the plants	backpressure upstream (steps 1–2)
a packer takes the next tray only when her bench is clear	a pull-based consumer (step 4)
the yard takes no more trays than the line packs inside four hours; the rest go to the jam lorry while still fresh	the deadline-sized bound, and turning the rest away (steps 5–6)
a tray already three hours old is not let in behind a two-hour line	admitting what can finish (step 7)
short-handed, the line drops jam fruit first, then farm-shop punnets, and keeps the supermarket and export orders	shedding from the bottom (steps 8–9)
when the yard is backed up, the packers take the freshest trays first; the oldest are past four hours anyway	newest first (step 11)

Breaks down when: late fruit still makes jam; a late response is worth nothing. And pickers can see the full bay, while a producer learns only from the signal it is given.

Contrasting case: the supermarket's weekly order, fixed in advance whatever the line can pack today: a rate limit, not backpressure (step 14).

In real terms: decide at the door what can be served while its caller waits, turn the rest away while they can still go elsewhere, and drop the least critical work first.

Examples, run for real

4.1 ExamHall EXH-2650: 09:00 answer saves behind an unbounded queue

ExamHall's answer-save service writes 400 saves a second, and a candidate's browser waits 2 s for a reply. At 09:00:00, 3,000 saves arrive together when a reconnect wave lands. Unbounded, the service writes every save in arrival order; MR !2950 caps it at 800 inside and turns the rest away with a 503 at once:

4.1 one writer, 400 saves a second · 3,000 saves at 09:00:00 · browsers wait 2 s · cap = 400 x 2 s = 800
4.1 unbounded, oldest first: in time 800 · written after the browser stopped waiting 2,200 · turned away at once 0 · last write at 7.5 s
4.1 MR !2950, cap 800 inside: in time 800 · written after the browser stopped waiting 0 · turned away at once 2,200 · last write at 2.0 s

Unbounded, the service drops nothing and still finishes only 800 in time; the rest wait, and it writes them after their browsers stopped waiting (steps 5 and 10). The cap of 400 × 2 s finishes the same 800 in time, and the other 2,200 hear at once that they may retry (steps 5–6).

4.2 ExamHall EXH-2650: criticality on the next hop

MR !2950 also tags traffic where it enters: answer saves CRITICAL_PLUS, paper fetches CRITICAL, proctor thumbnails SHEDDABLE_PLUS, analytics SHEDDABLE. Paper fetches and analytics both call the question bank, which serves 700 calls a second and sheds from the bottom:

4.2 question bank serves 700 calls a second · paper fetches call it 350 a second, analytics 650 a second
4.2 calls inherit their request's tag (CRITICAL, SHEDDABLE): paper fetches served 350 · analytics served 350
4.2 calls untagged, so shed evenly:                         paper fetches served 245 · analytics served 455

When each call inherits its request's tag, the question bank sheds analytics and serves every paper fetch. Untagged, every call looks alike and it sheds both evenly (steps 8–9).

Try it

Answer first, then check. Nothing is saved.

Question 1 of 7

A catalog's nightly feed job copies 1,000 product rows from a database cursor into an upload stream:

The database hands over rows four times as fast as the upload sends them, and the job's memory grows until it holds most of the feed.

Which change keeps no more than 16 rows written to out and not yet sent, and still uploads all 1,000?

const out = upload({ objectMode: true, highWaterMark: 16 });for await (const row of cursor) out.write(row);

Question 2 of 7

A ticket shop's seat-hold service has 8 workers and one queue; each hold takes 25 ms of work, and a free worker takes the next hold at once. The service caps how many holds may be inside, waiting or being worked on, and turns any hold over the cap away at once with a 503. At 10:00, 2,000 holds arrive together, then none for a minute. A browser waits 1.5 s for its hold, then stops waiting; the service does not check whether a browser is still waiting.

Which cap finishes the most holds while their browser still waits, with no hold finishing after its browser stopped waiting?

Question 3 of 7

A payments ledger's API can serve 1,500 requests a second, and every request carries a criticality tag set where it enters. At a month-end peak it receives 4,500 a second: postings tagged CRITICAL_PLUS at 400 a second, balance checks at checkout tagged CRITICAL at 500, statement downloads tagged SHEDDABLE_PLUS at 900, and reporting queries tagged SHEDDABLE at 2,700. Its overload handling follows the Google SRE book's criticality rules.

About how many statement downloads a second does it serve?

Question 4 of 7

A chat service's history API has one worker; each request takes 10 ms of work, and a caller waits 1 s. It serves its queue oldest first and rejects nothing. Two load tests: in the first, 20 requests arrive together, then 50 a second for eight seconds; in the second, 200 requests a second arrive for ten seconds. The team wants no request rejected in the first test, and in the second a median wait before work starts under 100 ms.

Which change meets both?

Question 5 of 7

A payments ledger's API is capped at 1,200 requests a second by a token bucket, a figure taken from last spring's stress test. The fleet now autoscales several times a day. Yesterday a slow fraud-check service made the API stop responding while traffic stayed under 1,200 a second.

Which limit design does the Netflix concurrency-limits README argue for here?

Question 6 of 7

A payments ledger imports a day's 600 settlement records from a card network. records is Readable.from(fetchRecords(), { highWaterMark: 10 }), where fetchRecords() is an async generator that asks the network's API for one record at a time and yields it. store(record) returns a promise that settles when that record is written to the ledger. The API hands over records three times as fast as stores finish.

Which loop keeps the import holding no more than about 10 records fetched but not yet stored, with one store at a time?

Question 7 of 7

A video platform's media API has one queue in front of one worker, served oldest first; each request takes 10 ms of work. Two kinds share it: thumbnail requests, whose player waits 2 s, and transcode-status requests, whose dashboard waits 10 s. The service does not check whether a caller is still waiting. At 20:00, 1,200 requests arrive together: 300 thumbnails and 900 status requests, every fourth one a thumbnail.

Which admission rule finishes the most requests while their caller still waits?

Go deeper

Pro tips

Set criticality in the first service a request reaches, let every call it makes inherit it, and graph rejections per criticality. Then an overloaded dependency deep in the stack sheds the same work the front door would (the SRE book's practice: "set the criticality as close as possible to the browsers or mobile clients"). When: a dependency shared by user-facing and background traffic (step 8). Status: RAN.

Source

Every graded script checks each hash and asserts the passages its rules come from before it runs. Valid as of 2026-10-02.

Google SRE book, Alejandro Forero Cuervo, "Handling Overload": https://sre.google/sre-book/handling-overload/ (sre-handling-overload.txt). The curriculum row's named source. Quoted: the four values CRITICAL_PLUS, CRITICAL, SHEDDABLE_PLUS and SHEDDABLE; "Services are expected to provision enough capacity for all expected CRITICAL and CRITICAL_PLUS traffic."; "a backend task will only reject requests of a given criticality if it's already rejecting all requests of all lower criticalities"; "When a task is itself overloaded, it will reject requests of lower criticalities sooner."; "request B and request C will use the same criticality as request A by default"; "set the criticality as close as possible to the browsers or mobile clients". Its retry budgets are ds-retries-backoff-jitter's.
Netflix, concurrency-limits README: https://github.com/Netflix/concurrency-limits (netflix-readme.txt, from the raw README). The curriculum row's named source; a project README the maintainers may change, so its claims are quoted as of 2026-10-02, the date read. Quoted: "in large distributed systems that auto-scale this value quickly goes out of date"; "Instead of thinking in terms of RPS, we should be thinking in terms of concurrent requests"; "Concurrency limits are very easy to enforce but difficult to determine"; "To estimate the limit we borrow from common TCP congestion control algorithms by equating a system's concurrency limit to a TCP congestion window."; "We can use latency measurements to determine when queuing happens."
Google SRE book, Mike Ulrich, "Addressing Cascading Failures": https://sre.google/sre-book/addressing-cascading-failures/ (sre-cascading.txt). Supporting, not named in the row, and keys nothing. Quoted: "Changing the queuing method from the standard first-in, first-out (FIFO) to last-in, first-out (LIFO) or using the controlled delay (CoDel) algorithm [Nic12] or similar approaches can reduce load by removing requests that are unlikely to be worth processing [Mau15]."; "you don’t get credit for late assignments with RPCs."; "it is usually better to have small queue lengths relative to the thread pool size (e.g., 50% or less)". Supports steps 5, 10–12.
IETF, Nichols et al., RFC 8289, Controlled Delay Active Queue Management: https://www.rfc-editor.org/rfc/rfc8289.txt (rfc8289.txt). Supporting, keys nothing. Quoted: "Instead of averages, we recommend tracking the minimum sojourn time; then, if there is one packet that has a zero sojourn time, there is no persistent queue."; the interval "is related to the amount of time it takes for the largest expected burst to drain." This lesson's CoDel-style rule borrows that idea for requests; it is not RFC 8289's packet algorithm, which also has a drop-rate control law. Supports step 12.
Node.js, "Backpressuring in Streams": https://nodejs.org/en/learn/modules/backpressuring-in-streams (node-backpressure.txt), and Stream API: https://nodejs.org/api/stream.html (node-stream.txt). Implementation references, not named in the row; the stream items are executed on Node 24.19.0 itself. Quoted: "The moment that backpressure is triggered can be narrowed exactly to the return value of a Writable's .write() function."; "If false is returned, further attempts to write data to the stream should stop until the 'drain' event is emitted."; "While a stream is not draining, calls to write() will buffer chunk, and return false." Supports steps 2–4.
Not fetched: Ben Maurer, "Fail at Scale", Communications of the ACM 58(11), 2015, the row's source for adaptive LIFO, CoDel and concurrency control. queue.acm.org, cacm.acm.org and dl.acm.org answered 403 to curl on 2026-10-02 (as on 2026-10-01, CL-030), so it quotes and keys nothing here; steps 10–12 rest on the runs, with the SRE chapter's sentence above as support.

Checked against the tools and docs as of 2026-10-02.

Predict first