Calibrated confidence language: word each claim as sure as its evidence

Word each claim as sure as what it rests on: no surer, and no less sure.

Before you read: predict

ExamHall mock round. The interviewer said 40,000 candidates sit the exam at once and the app autosaves each answer sheet every 30 seconds. You worked out 1,333 writes a second, multiplied by 3 for the peak, a factor you chose yourself, and said 4,000.

She asks: “How confident are you in that 4,000?” Which reply do you give?

The 30-second version

Your words state your confidence, and the interviewer can test them with one question. Each claim gets its own wording, chosen by what it rests on. Say your confidence before you answer, log it against the verdicts, and reword wherever the two part.

Word each claim as sure as what it rests on: no surer, and no less sure.

How it works

Step 1 of 14

Your wording is a claim. “I'm sure” promises that a follow-up won't move you; she can test that at once. [S1]

Analogy: The forensic glass examiner's report

A forensic scientist writes up one case. What she measured on her own bench, the refractive index of two glass fragments, she states plainly. What she infers from it carries one qualifier and its reason. What nobody has tested yet she marks as untested, with what she would expect. She never writes “could possibly perhaps” on a line; one phrase does that job. When a reviewer asks “are you sure?”, she goes back to her bench notes. Each year the lab's proficiency tests set her past wordings against the true answers.

In the report	In the interview
A measurement stated plainly	A checked claim, said plainly
An inference with one qualifier and its reason	An inference at the middle rung, with its reason
“Untested; I would expect…”	A guess at the “need to check” rung, with what you'd expect
Back to the bench notes	“Are you sure?” answered from your basis
Proficiency tests against the true answers	Your log against the verdicts

Breaks down when: her words come from a fixed scale and she has days to choose them; you choose yours aloud in seconds. A report is also read once, while an interviewer can ask again at once.

Examples, run for real

EXH-4530 is a synthetic ExamHall mock on the row's hook: before each of 20 answers the candidate said “sure”, “fairly confident” or “need to check”, then saw the verdict. MR !4830, her next mock, keeps “sure” for what the question itself stated.

EXH-4530 mock 1, the confidence tap said aloud · 20 answers · 13 right overall
  said "sure" 10 times · right 7 · 70% · in the "fairly confident" band · worded above it
  said "fairly confident" 6 times · right 4 · 67% · in the "fairly confident" band · matches
  said "need to check" 4 times · right 2 · 50% · in the "need to check" band · matches
MR !4830 mock 2, "sure" kept for what the question stated · 20 answers · 15 right overall
  said "sure" 8 times · right 8 · 100% · in the "sure" band · matches
  said "fairly confident" 7 times · right 5 · 71% · in the "fairly confident" band · matches
  said "need to check" 5 times · right 2 · 40% · in the "need to check" band · matches

Her “sure” was right 7 times in 10: a “fairly confident” record, worded a rung too high (step 13). Her overall 13 of 20 hides which rung is off; only the rung-by-rung reading shows it. In the next mock she said “sure” only where the question gave her the fact, and every rung sits in its band. The log checks how she worded her answers, not how much she knew.

Try it

Answer first, then check. Nothing is saved.

Question 1 of 7

Ticketing round, on seat holds for a concert on-sale. You read the hold service's config, which the interviewer shared: holds last 10 minutes, and a release job runs every minute. You have no figure for how many holds go unpaid; your own guess is 1 in 5.

Which version do you say?

Question 2 of 7

Three ticketing mock rounds. Before each answer you said your confidence; afterwards you logged the verdicts. “Sure”: said 11 times, right 10. “Fairly confident”: said 9 times, right 6. “Need to check”: said 8 times, right 7.

Read with step 12's rule of thumb, which change does the log support?

Question 3 of 7

Ticketing round. The interviewer shared the hold service's code, and you read HOLD_TTL_SECONDS = 600 in it.

You said: “A seat hold lasts 10 minutes.” She asks: “Are you sure?” What do you say?

Question 4 of 7

Ticketing round, on paying for tickets. You said the payment provider retries a declined card three times. Two years ago, at another job, you used this provider's API and saw it retry three times; you haven't seen its documentation since.

The interviewer asks: “Is that how it works, or are you guessing?” What do you say?

Question 5 of 7

One line for each rung, in the interviewer's on-sale case. Sure: the venue's floor plan, which she showed you, has 20,000 seats. Fairly: from her figures (60,000 fans queuing, a limit of 4 tickets each) you infer that the seats sell out before the queue is through. Check: you have no data on how many fans in the queue are bots; give the share you'd expect.

Write the three lines, each starting with its label.

Your answer (optional, stays on this page)

Question 6 of 7

Ticketing round, a week later, on a big on-sale's queue. You said five things. (1) “I'm sure the queue lets in 500 fans a second.” The interviewer read that figure out to you from the live queue's dashboard. (2) “Most fans give up after 20 minutes in the queue.” Nothing you were shown says so. (3) “I'm fairly confident about a third of the queue are bots, because on-sales attract a lot of them.” That is your impression; you have no figure. (4) “I'm fairly confident the payment step is the bottleneck, because its p95 in the log you showed is 4 seconds and every other step is under 200 ms.” The log showed those figures and named no bottleneck. (5) “Each fan can maybe buy up to 4 tickets, I think.” The interviewer told you the limit is 4.

Which of the five did you word above what they rest on?

Question 7 of 7

Ticketing round, a week later. The interviewer showed you the hold service's config; it gives the hold's length, 10 minutes, and nothing else.

You said: “Holds are definitely released the second they expire.” She asks: “Are you sure?” What do you say?

Go deeper

Pro tip

For one week of practice, say a rung out loud before every answer, even to yourself, and tick it against the answer key afterwards. At the end of the week, count each rung's ticks. The rung you trust least is usually the one to change.

Checked against the tools and docs as of 2026-10-15.

Predict first