The verification loop for generated code

Check generated code against the brief, on your own stack, before you accept it.

Before you read: predict

ExamHall's brief for a new admin-console rule: "A late submission loses 10 points for each started hour after the deadline, and its score never drops below 0." The candidate asks the assistant for it, and it returns this function with four tests:

// late.test.mjs
it('keeps an on-time score', () => expect(lateScore(72, 0)).toBe(72));
it('takes 20 for two hours late', () => expect(lateScore(72, 120)).toBe(52));
it('never goes below zero', () => expect(lateScore(15, 600)).toBe(0));
it('keeps the score when just late', () => expect(lateScore(72, 30)).toBe(72));

She runs them: 4 passed.

Before she accepts the diff, what does this run show?

// late.mjsexport function lateScore(score, minutesLate) {  if (minutesLate <= 0) return score;  const hours = Math.floor(minutesLate / 60);  return Math.max(0, score - 10 * hours);}

The 30-second version

An assistant's code is fluent and usually runs on its own example, so a green first run says little. Treat each diff as an untrusted change. Read it, then run tests whose expected values you worked out from the brief, in the project, on its installed versions, error paths included. After any rewrite, rerun them all; only then accept.

Check generated code against the brief, on your own stack, before you accept it.

How it works

Step 1 of 12

Analogy: A newspaper crossword's test-solver

Before a new crossword is printed, the editor sends it to a test-solver. The setter's grid arrives with an answer sheet. The test-solver puts the sheet face down and solves from the clues alone, then compares. Where her answer and the sheet differ, the clue decides: the sheet only records the setter's own reading. She looks hardest at the squares where two answers cross, and every answer must be in the dictionary edition the paper uses now. When the setter rewrites one answer, she solves the whole grid again, because the crossing letters move.

At the crossword desk	With a generated diff
The setter's answer sheet	The assistant's own tests
Solving from the clues alone, sheet face down	Expected values worked out from the brief
Squares where two answers cross	Cases where two of the brief's rules meet
The paper's current dictionary edition	The project's installed versions
The setter says "I've checked it"	The assistant says "no bugs found"
Re-solving the grid after one answer changes	Rerunning every test after a regenerated fix

Breaks down when: a grid is finite and every square can be checked, while code meets inputs nobody listed. Wrong inference: "once each brief test passes, every input is safe." A test-solver also works alone, while in the round you verify with the interviewer watching.

In real terms: before you accept, the brief's cases have run, on your stack, and passed.

Examples, run for real

4.1 The countdown that passed its example (EXH-2240)

EXH-2240 is a synthetic generated diff for the ExamHall admin console's exam countdown, the case in this concept's curriculum row. Asked for "a countdown from 90 minutes that updates every second", the assistant returned a function that subtracts one second per timer callback.

4.1 EXH-2240 generated countdown · the prompt's example (start of a 90-minute exam): shows 90:00 · passes
4.1 visible tab, clock moved 3 s, 3 callbacks: shows 89:57 · clock says 89:57 · passes
4.1 hidden tab, clock moved 300 s, callbacks held (model) and 1 on return: shows 89:56 · clock says 84:57 · fails
4.1 MR !2540 redraws from the deadline: hidden-tab test shows 84:57 · clock says 84:57 · passes

The prompt's example and a three-second check both pass, because each callback arrived. Only a test that controls the clock and holds callbacks, as a background tab may, reaches the case the brief cares about: candidates switch tabs during an exam. Why counting callbacks drifts is fsd-clock-sync-timers' topic; here the point is which check reached the failure. MR !2540 redraws from the deadline, and the same test passes.

4.2 A helper written for another Node version

A generated checkout helper formats the delivery day and checks its input with util.isDate. It reads correctly, and it ran on the Node versions the model learned from.

4.2 node 24.19.0 · typeof util.isDate: undefined
4.2 deliveryDay('2026-11-03T10:00:00Z') -> TypeError: util.isDate is not a function
4.2 with input instanceof Date in its place -> 2026-11-03

Node.js removed util.isDate() in v23.0.0 and suggests arg instanceof Date instead (S3). Reading the helper shows nothing wrong; running it on the project's Node does. Which APIs a version removed is aiassist-rejecting-wrong-suggestions' topic; this lesson's point is where the check runs.

Picture it

A woman at a library desk holds two open books side by side, comparing them; their pages are blank, and two more books lie on the desk. — An illustration to help it stick. The mechanism is in the sections above.

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

The checkout API's coupon handler must answer a valid coupon with status 200 and the coupon, and an invalid one with status 400 and the names of the fields that failed. The assistant wrote:

The project's lockfile pins zod 4.6.5. The candidate's scratch folder, where she tries snippets, has zod 3.25.76.

Which one check, run before she accepts, shows the handler failing its brief?

// coupon.mjsimport { z } from 'zod';const Coupon = z.object({ code: z.string().min(3), percent: z.number().int().max(50) });export function checkCoupon(body) {  const r = Coupon.safeParse(body);  if (!r.success) return { status: 400, fields: r.error.errors.map((e) => e.path.join('.')) };  return { status: 200, coupon: r.data };}

Question 2 of 5

In Meta's AI-enabled coding round, a candidate pastes the assistant's applyCoupons into the cart module and types in its chat: "Any bugs in this?" It replies: "No bugs found. It handles empty carts, stacked coupons and rounding." The test runner shows no run since the paste, and he moves on to the next part.

As Hello Interview describes the round as of 2026-10-01, how much does that reply count as checking the code?

Question 3 of 5

The brief: "Standard delivery is free when the items come to £50 or more, and costs £4.99 otherwise. Each bulky item adds £10, whatever the basket." The candidate wrote four tests from the brief:

The assistant's first version failed only the last test. She pasted that test into the chat, and it sent a new version, "now adding £10 per bulky item on free-delivery baskets too", with two tests of its own:

// shipping.mjs
export function shippingFee(itemsTotal, bulky) {
  if (bulky > 0) return 10 * bulky;
  return itemsTotal >= 50 ? 0 : 4.99;
}

// shipping.assistant.test.mjs
it('one bulky item on a large basket', () => expect(shippingFee(60, 1)).toBe(10));
it('three bulky items on a large basket', () => expect(shippingFee(90, 3)).toBe(30));

Which one re-run, on the new version, shows that it is not ready to accept?

// shipping.test.mjsit('small basket pays standard', () => expect(shippingFee(20, 0)).toBe(4.99));it('large basket ships free', () => expect(shippingFee(80, 0)).toBe(0));it('small basket with a bulky item', () => expect(shippingFee(20, 1)).toBe(14.99));it('large basket with two bulky items', () => expect(shippingFee(80, 2)).toBe(20));

Question 4 of 5

The brief: "A percentage coupon comes off the price of the basket's items, not off any gift cards in it, and the discount is rounded down to the penny." Each basket line is { price, qty, kind }, with kind either 'item' or 'giftcard'. The assistant wrote:

Which basket, tested with the discount worked out from the brief, shows this code failing it?

// discount.mjsexport function couponDiscount(lines, percent) {  const total = lines.reduce((sum, l) => sum + l.price * l.qty, 0);  return Math.floor(total * percent) / 100;}

Question 5 of 5

A candidate rehearses for Meta's AI-enabled round on a checkout pricing task. Her plan: ask the assistant for four helpers one at a time, read each one closely as it lands, paste it in, and run the test suite once, after the fourth.

As Hello Interview describes what interviewers want to see in that round, as of 2026-10-01, what should she change?

Go deeper

Pro tip

Before you prompt, write two or three of the brief's cases as test lines with their expected values. When the code arrives, they run first.

Checked against the tools and docs as of 2026-10-01.

Predict first