Fixture makes both branches identical

What goes wrong

The setup data is chosen so that the correct path and the wrong one return the same answer, so no assertion can tell which of them ran. The suite stays green either way, and the reader takes that green result as proof the intended path was taken.

Example

A test proves a value is handed down from the settings object, but the fixture sets that setting to the same number the built-in default already gives.

Not to be confused with

Not an assertion that cannot fail, where the check itself is vacuous; here the check is sound and it is the chosen data that erases the difference the check would otherwise catch.

Discovered by

@sks24git GitHub

Field notes · 20

@sks24git2026-11-06beat 1 review round

A test that proves a report writes nothing counted stored rows only after the report had run twice. A write to a fixed key inserts once and then only updates, so the count never moves and the test stays green.

Snapshot before the first call. A baseline taken after the code ran already holds its writes.
Compare row contents, not counts. An update keeps the count and changes the row.

@sks24git2026-11-06beat 3 review rounds

A test of a time cutoff moved its "too late" row into the future, so a cutoff at the present moment rejected it too. Swap the real cutoff for "now" and every test stayed green while the job never ended.

Put the edge case between the two cutoffs you must tell apart. A row past both proves neither.
Mutate each guard once. A test that stays green proves nothing.

@sks24git2026-11-05

A test meant to prove a wait reads live state used a fixture that settled through a different branch from the first read. The live-state code could be deleted and the test stayed green, claiming coverage it never had.

Build the fixture only the code under test can settle. Any other exit makes the test blind.
Name in a test only what its fixture forces. An overclaiming message misleads the next reader.

@sks24git2026-11-05

A test fixture set one slow fade above the element, so waiting on the parents alone already covered the fade inside it. Delete the inside check and the test still passes, so it proves nothing about that branch.

Make the branch under test the last thing to finish. If another one ends later, it hides the gap.
Use one fixture per mutant when timings clash. One page can't let each branch be the longest.

@sks24git2026-11-04

Calls made parallel to beat a deadline were tested with a stub that answers at once, so a plain one-by-one loop passes every test. Ship that loop by mistake and real calls run past the deadline on any large batch.

Give the stub a delay when concurrency is the point. An instant stub cannot tell parallel from serial.
Assert the overlap, not just the result. Check total time stays below the sum of calls.

@sks24git2026-11-04

A test fed a setting far above its range and expected the default, but the default and the range's top were the same number. Swap the fallback for a clamp to the top and the test stays green.

Probe a fallback from the side where the answers differ. When two outcomes share a value, the test cannot tell them apart.

@sks24git2026-11-04beat 2 review rounds

A test fakes a write failure by throwing in the app, so the database never sees an error. Move the write back inside the transaction and the test stays green, while a real error there rolls back the user's saved work.

Inject failures where the guarded effect happens. A fake error above the database cannot show what the database does with a real one.

@sks24git2026-11-04beat 1 review round

A data copy test seeded its only out-of-scope row for a user whose target value already existed, so the skip-on-conflict hid it. Drop the scope filter and the test stays green while every user gets a wrong value.

Seed each filter's decoy where nothing else can mask it. One guard covering another lets a deleted filter pass unseen.

@sks24git2026-11-04beat 2 review rounds

A dedupe memory reads the message text, falling back to the caption, but no test edits a file's caption to a new value. Drop the fallback and every caption fix looks unchanged, so the client's correction vanishes.

Test every fallback branch with a value that differs. Two empty strings always compare equal.
Fill test state the way production does. Hand-seeded state hides the real write path.

@sks24git2026-11-03beat 2 review rounds

A test fakes a lock failure before any statement runs, so the database never sees a broken transaction. Remove the rollback and every test passes, while a real timeout hands a dead connection back to the pool.

Fail the real statement, not a fake before it. A fake error skips the state the cleanup exists for.

@sks24git2026-11-03

A merge that keeps the larger of two values was tested with one pair, and the incoming value was the larger in it. Swap the rule for "incoming always wins" and the test stays green while real merges drop the bigger one.

Test a max-merge from both sides. One case where the new value wins cannot tell max from overwrite.

@sks24git2026-11-03beat 1 review round

A test checked that unknown commands change nothing, but it started from the very state a broken command map falls back to. Map every unknown command to that state and the test stays green.

Start a "nothing changes" test from a state the fallback would overwrite. If the fallback lands where you began, the check cannot fail.

@sks24git2026-10-30beat 4 review rounds

A setting passed through to a command was tested with one value only, the same one a hard-coded constant gives. Swap the variable for that constant and tests pass while the other value falls back to old behaviour.

Test a passed-through value with two settings. One setting cannot tell a variable from a constant.

@sks24git2026-10-29beat 1 review round

A test pins a retry branch name built from the round and the attempt, and the fixture sets both to the same number. Swap one for the other in the code and the name comes out the same, so the test stays green.

Give each fixture value its own number. Two fields set alike let a mix-up pass.
Check that a test fails on the mutant you fear. A green test only means something if it can go red.

@sks24git2026-10-29beat 3 review rounds

A config object passes through a helper, and the one test that drives it uses values equal to the defaults. Drop all fields but one and every test stays green while production loses its patience.

Test a pass-through with non-default values. A value equal to the default cannot tell forwarding from fallback.

@sks24git2026-09-17beat 4 review rounds

A date came from the scope's own time zone, but no test wired the zone provider, so code and tests both fell back to UTC. Swap the zone lookup for a plain UTC date and the suite stays green.

Wire the dependency the branch exists for. A test on the fallback proves the fallback, not the branch.
Assert the premise. Pin that local date differs from UTC, else oracle equals mutation.

@sks24git2026-09-13beat 1 review round

An assertion accepted either of two box names, and in the fixture both boxes had the same edge. Drop one from the collector and the check still passes, so half of what it reads stops being read.

An alternation is satisfied by its easiest branch. Assert the one the comment names.
Make the fixture separate what you alternate. Equal boxes hide which branch fired.

@sks24git2026-09-12beat 1 review round

A fixture grew so both tables had a second page, closing one blind spot. It opened another: the flag telling each pager when to stop now read the same on both sides, so swapping them changed nothing.

Re-run every mutant after editing a fixture. A fixture change can retire a check that used to fail loudly.
A fix to a test is a change to your coverage.

@sks24git2026-09-09

A reader was split into two streams, a byte tail and the last complete record, so a cap check would see a whole line. No test told them apart, so pointing the check at the wrong stream stayed green.

A split with no distinguishing test is not a split. Feed each stream a value the other would misread.

@sks24git2026-09-08beat 5 review rounds

A cheap guard skipped an expensive probe on the path where its answer would be a lie. No test noticed the guard: every fixture that reached that path made the probe return nothing anyway.

Pin the skip, not just the result. A guard nobody can turn red is a comment, not code.
Fixtures that stay silent hide mutations. Make the fixture answer, so the wrong answer shows.