Skip to content

Coverage vs mutation testing

Coverage shows what your tests ran; mutation testing shows what they would catch.

Predict first

Before you read: predict

ExamHall's scoring module and its two tests:

// score.test.mjs
it('grades a strong paper', () => { grade(90, 100, 70); });
it('fails a weak paper', () => { expect(grade(40, 100, 70)).toBe('fail'); });

Vitest measures line and branch coverage of score.mjs. Stryker then mutates line 3 only. Its five mutants replace the condition with true, false, pct > passMark and pct < passMark, and replace 'pass' with ""; it runs both tests against each mutant.

What do the two reports show?

Code for this question
// score.mjsexport function grade(points, max, passMark) {  const pct = Math.round((points / max) * 100);  if (pct >= passMark) return 'pass';  return 'fail';}
Pick one

The 30-second version

Coverage reports which lines and branches the tests ran, not whether any assertion would notice a fault. Mutation testing plants one small fault at a time and reruns the tests: a failing test kills the mutant, an all-green run lets it survive. The mutation score is the share detected; each survivor points at a missing input or assertion.

Coverage shows what your tests ran; mutation testing shows what they would catch.

How it works

Step 1 of 11

Line coverage counts the lines a test run executed. A line counts as covered once any statement on it has run.
In the kitchen
  • the belt counter logs every bag through the head → line coverage: the code ran (step 1)

Analogy: The metal detector on a crisp factory's bagging line

Every bag passes through the detector head, and the belt counter logs each one. Each hour a quality technician tapes a certified metal test piece into a bag and sends it through.

On the bagging line In the test suite
the belt counter logs every bag through the head line coverage: the code ran (step 1)
a head turned to its lowest sensitivity still lets the counter tick a test with no assertion still adds coverage (step 2)
a day when no bag set the detector off never moved the reject arm a covered line with an untaken branch (steps 3–4)
the test piece's bag is rejected a mutant killed (step 6)
the test piece rides through unrejected a survivor: the head cannot see that fault (step 9)
a test piece sent down the side lane that bypasses the head no coverage (step 7)
test pieces rejected over test pieces sent the mutation score (step 8)

In real terms: coverage counts what the tests executed; the mutation score counts the planted faults the tests' failures exposed.

Breaks down when: every test piece is a real contaminant, but some code mutants change nothing any input can show (equivalent mutants, step 10). And a detector inspects every bag the same way, while each test checks only what its assertions read.

Contrasting case: a code reviewer reading the diff finds faults without running anything. Mutation testing measures the suite, not the code.

Examples, run for real

4.1 ExamHall: the pass-mark refactor (EXH-2110)

EXH-2110: a refactor turned pct >= passMark into pct > passMark, so candidates exactly on the mark were failed. The suite stayed green. Its tests graded a 90 ('pass') and a 40 ('fail') against a pass mark of 70.

4.1 coverage   score.mjs lines 100% (3/3) · branches 100% (2/2)
4.1 stryker    line 3: 5 mutants · killed 4 · survived 1: pct > passMark
4.1 MR !2410   + expect(grade(70, 100, 70)).toBe('pass') · line 3: killed 5 · survived 0

Full coverage, yet one mutant lives: no test separates >= from >, because the two differ only on the mark itself (steps 6, 9). MR !2410 adds the paper on the mark, and Stryker now kills all five.

4.2 Proof domain, checkout: a fallback no test reached

The checkout shows a delivery label, falling back to a default when the slot has none. Its one test passes a slot label.

export function deliveryLabel(slotLabel) {
  return slotLabel ?? 'Standard delivery';
}
4.2 coverage   delivery.mjs lines 100% (1/1) · branches 50% (1/2)
4.2 stryker    killed 2 · survived 0 · no coverage 1: 'Standard delivery' -> "" · score total 66.67

Every line ran, but the ?? fallback never did (steps 3–4). Stryker tracks coverage per mutant, not per line, so the default string's mutant gets No coverage on a fully covered line. It still counts as undetected: two of three valid mutants, 66.67 (steps 7–8).

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

The checkout's shipping rule and its tests:

// shipping.test.mjs
it('ships a large basket free', () => { expect(shippingFee(80, 'GB')).toBe(0); });
it('charges a small basket', () => { expect(shippingFee(20, 'GB')).toBe(4.99); });

What line and branch coverage does Vitest report for shipping.mjs?

Code for this question
// shipping.mjsexport function shippingFee(subtotal, country) {  if (country !== 'GB') return 12;  return subtotal >= 50 ? 0 : 4.99;}
Pick one

Question 2 of 5

The coupon rule and its tests:

// coupon.test.mjs
it('takes ten percent off', () => { expect(couponValue(200, 'SAVE10').discount).toBe(20); });
it('ignores an unknown code', () => { expect(couponValue(200, 'NOPE').discount).toBe(0); });

Stryker's report includes this survivor on line 2: the first label string replaced with "".

Which one added test kills it?

Code for this question
// coupon.mjsexport function couponValue(total, code) {  if (code === 'SAVE10') return { discount: total * 0.1, label: 'Ten percent off' };  if (code === 'FREESHIP') return { discount: 0, label: 'Free delivery' };  return { discount: 0, label: '' };}
Pick one

Question 3 of 5

The refund rule and its tests:

// refund.test.mjs
it('refunds a damaged order in full', () => { expect(refundAmount({ total: 80, shipping: 4 }, 'damaged')).toBe(80); });
it('refunds nothing for a change of mind', () => { expect(refundAmount({ total: 80, shipping: 4 }, 'changed mind')).toBe(0); });

Stryker's run on refund.mjs ends with 7 mutants killed, 2 survived, 1 with no coverage, no timeouts and no errors.

What does it print as the file's total mutation score?

Code for this question
// refund.mjsexport function refundAmount(order, reason) {  if (reason === 'damaged') return order.total;  if (reason === 'late') return Math.min(order.shipping, 5);  return 0;}
Pick one

Question 4 of 5

The final-score rule and its tests:

// final.test.mjs
it('caps at 100', () => { expect(finalScore(95, 10, false)).toBe(100); });
it('adds the bonus', () => { expect(finalScore(40, 5, false)).toBe(45); });
it('rejects a big bonus', () => { expect(() => finalScore(50, 25, false)).toThrow(); });

The specification: raw is a whole number from 0 to 100, bonus a whole number from 0 to 30, late true or false, and a bonus over 20 throws RangeError('bonus over 20'). Stryker leaves 5 mutants undetected, these three among them.

Which of the three can no test written to this specification kill?

Code for this question
// final.mjsexport function finalScore(raw, bonus, late) {  if (bonus > 20) throw new RangeError('bonus over 20');  const s = raw + bonus;  if (late) return Math.floor(s / 2);  if (s >= 100) return 100;  return s;}
Pick one

Question 5 of 5

The checkout's loyalty-points service has 92% line coverage, above the release gate of 80%, and its lead proposes raising the gate to 95%. Last month a bug that expired members' points a day early shipped through its suite.

What should the team do with the service's coverage numbers?

Pick one

Go deeper

Pro tips
  1. Mutate only the lines a change touches. Give Stryker a line range in mutate, so a review-time run covers the diff rather than the whole module.
    npx stryker run --mutate "src/score.mjs:3-3"
    When: reviewing a change to a large module, where a whole-file run is slow and its report too long to act on. Verify: tip whole file: 9 mutants on lines 1,2,3,4 · --mutate "src/score.mjs:3-3": 5 mutants on line 3. Status: RAN.
Source

Tool behaviour is version-specific; the pack ran Vitest 5.0.2, @vitest/coverage-istanbul 5.0.2 and Stryker 10.0.0. Valid as of 2026-10-01.

Related lessons

Checked against the tools and docs as of 2026-10-01.