Skip to content

Consensus and quorums (awareness)

Count votes against the whole membership: only a majority decides, so a cut has at most one winning side.

Predict first

Before you read: predict

On this lesson's cluster model, a payments ledger keeps one primary database node, chosen by a small cluster. Its payment servers run in zones A and B, and each sends 1 payment a second to the primary its zone can reach. One afternoon zone A, which holds the current primary, is cut off from the other zones for 5 minutes; every node keeps running. In the voting designs, a node that hears no primary for 10 s calls an election.

Which design keeps payments being accepted through those 5 minutes, but never on two primaries at once?

Pick one

The 30-second version

Consensus lets a group of servers agree on one leader or one order of writes, and keep agreeing while some fail. Every decision needs a majority of all members, so the two sides of a cut can never both decide. You rarely write it yourself: you use it through a store such as etcd, or your database's failover tooling.

Count votes against the whole membership: only a majority decides, so a cut has at most one winning side.

How it works

Step 1 of 13

Consensus. Servers agree on a value, such as who leads or the next write; once decided, it is final. Raft

Analogy: A union branch's officers voting by phone

A delivery firm's union branch has officers at two depots and at the town office. Its rulebook lets the branch answer management only on a vote of a majority of all its officers, counted on the branch list.

At the union branch In a system
the vote counts against everyone on the branch list, absent officers included a quorum of all members (step 2)
two calls cannot both hold a majority of one list one winner (step 3)
with too few officers on the line, nothing is answered, and nothing wrong is said no quorum: the cluster waits (step 4)
the line to one depot is cut; its officer is silent, not gone a cut, not a crash (step 10)
each chair is elected for a numbered year; last year's chair stands down once told of the new one terms (steps 7–8)
taking on a new officer is itself put to the current list membership changes (step 12)
the regional office counts the branch's ballots a store such as etcd (step 13)

In real terms: whoever may act for the cluster is decided by a majority of everyone on the list, never by whoever is still on the line.

Breaks down when: cut-off officers can ring round later and agree one answer, while a cut-off server keeps acting on its old view until a newer term reaches it (step 8).

Contrasting case: a branch with one officer needs no vote: she answers or she is silent. One database node never has two primaries either; it simply stops when it fails.

Examples, run for real

4.1 ExamHall: two primaries after a cut, then a three-member vote

EXH-4740: ExamHall's answer store ran Postgres on two nodes, in zones A and B, and the standby promoted itself after 30 s without the primary's heartbeat. Zone A was cut off from the other zones for 4 minutes while both nodes kept running. MR !5040 lets three etcd members, one per zone, decide which node is primary.

4.1 EXH-4740 two Postgres nodes, promote after 30 s of silence · zone A cut off 240 s · primaries at once 2 · answers saved: zone A 2,880, zone B 2,520, on two copies that no longer agree
4.1 MR !5040 three etcd members, one per zone, decide the primary · same cut · primaries at once 1 · answers saved: zone A 0 (refused), zone B 2,760 · one copy

Both nodes saw silence and both acted as primary, so the answers now sit on two copies that disagree (steps 9–10). With three voters, zone A held 1 of 3 votes and refused; zones B and C held 2 and elected one primary (steps 2–3).

Try it

Answer first, then check. Nothing is saved.

Question 1 of 7

A payments ledger's primary is chosen by a 5-member etcd cluster. A teammate proposes adding a sixth member "for safety".

How many members can fail at once in each cluster while it can still elect a primary?

Pick one

Question 2 of 7

On this lesson's cluster model, a payments ledger's 5-member cluster has 2 members in zone A, 2 in zone B and 1 in zone C, and the leader is in zone A. A follower that hears no leader for 10 s calls an election, and messages between members that can reach each other arrive within the same second. Payment servers in zones A and B each send 1 payment a second to the leader their zone can reach. Zone A is cut off from zones B and C for 2 minutes; every member keeps running.

How many payments does the ledger commit during those 2 minutes?

Pick one

Question 3 of 7

A payments ledger's 3-member etcd cluster lost its zone C member with that member's disk; the other two are healthy. A new VM in zone C will replace it, and nobody yet knows whether its address is configured right.

Which order does the etcd FAQ give for the swap?

Pick one

Question 4 of 7

A payments ledger's etcd cluster is being placed across two data centres, DC1 (beside the payment servers) and DC2, and a small third site.

Which placement still elects a primary whichever one of the three sites goes down?

Pick one

Question 5 of 7

A teammate plans the ledger's failover as two Postgres nodes, one per data centre, each running a script that pings the other every second and promotes itself after 10 s of silence. Write three sentences to the teammate: what goes wrong when the link between the data centres fails, what to add and where, and what to use instead of the script.

What do you write to the teammate?

Question 6 of 7

On this lesson's cluster model, a payments ledger's 3-member cluster has one member in each of zones A, B and C, and the leader is in zone C. A follower that hears no leader for 10 s calls an election. Payment servers in zones A and C each send 1 payment a second to the leader their zone can reach. Zone C is cut off for 90 s; every member keeps running.

When the cut heals, which of the payments sent during the cut does the ledger hold?

Pick one

Question 7 of 7

A payments ledger's 5 etcd members sat in two data centres and a small third site. To save money the company drops the third site: the 5 members now sit 3 in DC1 and 2 in DC2.

Which data-centre failure does the cluster survive, still electing a primary?

Pick one

Go deeper

Pro tip
  1. Next to every cluster in a design, write its quorum, the failures it survives, and what each zone's loss leaves. When: sizing or placing etcd, ZooKeeper or a database's voting members (steps 2, 4, 11). Verify:
tip ExamHall's 3 etcd members, one per zone: quorum 2 · survives 1 failure · each zone down leaves 2 of 3
Sources
  • The Raft Consensus Algorithm: https://raft.github.io/ · raft.txt. Named in the curriculum row. "Consensus involves multiple servers agreeing on values. Once they reach a decision on a value, that decision is final."; "If more servers fail, they stop making progress (but will never return an incorrect result)." Supports steps 1 and 4. Valid as of 2026-10-17.
  • Diego Ongaro and John Ousterhout, In Search of an Understandable Consensus Algorithm (Extended Version), linked from raft.github.io as "the Raft paper": https://raft.github.io/raft.pdf · raft-paper.txt. "A candidate wins an election if it receives votes from a majority of the servers in the full cluster for the same term."; "The majority rule ensures that at most one candidate can win the election for a particular term"; "A log entry is committed once the leader that created the entry has replicated it on a majority of the servers"; "If a follower receives no communication, it becomes a candidate and initiates an election."; "If a candidate or leader discovers that its term is out of date, it immediately reverts to follower state." Supports steps 2–3 and 6–8 and the model's truth rule. Valid as of 2026-10-17.
  • etcd v3.6 documentation, Frequently Asked Questions: https://etcd.io/docs/v3.6/faq/ · etcd-faq.txt. Named in the curriculum row. "An etcd cluster needs a majority of nodes, a quorum, to agree on updates to the cluster state. For a cluster with n members, quorum is (n/2)+1."; "These proposals include key-value updates and membership changes."; "Raft is leader-based; the leader handles all client requests which need cluster consensus." Supports steps 2, 12 and 13. Valid as of 2026-10-17.

Related lessons

Checked against the tools and docs as of 2026-10-17.