Documented guarantee not kept by code

What goes wrong

A comment, doc or runbook promises how the code behaves — 'retries three times', 'called exactly once', 'always runs on the main thread' — and the code does not do that. A caller or operator relies on the promise.

Example

The doc comment on a loader says 'idempotent, safe to retry', but a retry after a half-finished run inserts every row a second time.

Not to be confused with

Not a stale list of participants (Stale consumer list in comment), not a stale file:line link, and not two instruction files that contradict each other for an AI agent (Two AI-agent rule files disagree).

Discovered by

@sks24git GitHub

Field notes · 236

@sks24git2026-11-06beat 3 review rounds

A list of rewards said stories about any bug count, but the rule only counts stories about serious bugs. Users would post stories about small bugs and wait for a reward that never comes.

Copy every condition of an offer, not its headline. The fine print is what the grant checks.
Write one sentence for a rule and reuse it. Two pages describing one rule drift apart.

@sks24git2026-11-04

A change added an index, but two comments elsewhere still said nothing indexes that column. The next reader trusts the comment, reasons about a full scan that no longer happens, and tunes the wrong thing.

Grep for claims of absence when you add something. "Nothing uses X" goes false the moment you add X.
Prefer comments that say why over what exists. Inventories rot first.

@sks24git2026-11-04

Round 3 of tick diagnostics: two mediums came from my own PR text promising a 5 s fallback the code never had. Graded by consequence: both low; merging, lows to debt.

Write the PR body from the code, not from memory: a wrong description becomes findings.

@sks24git2026-11-03

A page told users a step was optional, while the form under it still required every field and the server refused a partial post. Users who trust the line get blocked or bounced back.

Change the words and the form together. A promise the form will not keep is a bug.
Render the page in the test, not only the count. A label nobody renders breaks silently.

@sks24git2026-11-02beat 1 review round

A launch claim lost its "not yet true" tag because the feature's code had merged. But that code only runs as an off-by-default experiment, so the public video promises something no real user ever gets.

Clear a claim on what production runs, not what merged. A flag or experiment arm can keep shipped code dark for every user.

@sks24git2026-11-01beat 1 review round

A feature was cut back to a few pages, but the help text still said every page had it. Readers from the other pages follow it to an option that does not exist, and no test reads help text against the code.

When you narrow a feature, grep the docs in the same commit. Help text outlives the code.
Test the claim, not just the widget. A green UI check says nothing about the prose.

@sks24git2026-11-01beat 4 review rounds

A new reason to hold back an offer told the client that asking for it anyway would still work. Other checks further down could still refuse that request, so the client promised the user something it could not deliver.

Promise only what the next check cannot take back. A reason that answers early hides the ones after it.
Fix a false sentence by deleting it. A truer rewrite tends to grow a new false claim.

@sks24git2026-10-31beat 5 review rounds

Help pages promised that the offer would say which of two prices applies, but the offer only listed both. A reader who saw no warning about the higher price could still be charged it.

Make public text claim only what the screen shows. When the screen changes, reread every page that promises something about it.

@sks24git2026-10-28beat 2 review rounds

A design doc described a cleanup rule as its authors remembered it, not as the code branches, so a simulation built on it would free resources the real system never frees. Its replay would explain the wrong night.

Cite the branch, not the constant. Thresholds that match the code can still describe behaviour the code never executes.

@sks24git2026-10-26

A runbook claimed a field could not be reset because sending null or an empty string left it unchanged. The API had a separate clear flag, so operators deleted and recreated the record and broke everything linked to it.

Grep the update schema before writing "cannot". Null-means-absent APIs often hide an explicit clear flag.
Fix records in place. Delete-and-recreate orphans every link to the old id.

@sks24git2026-10-26

A knob's help text promises tickets near a deadline go first, but the code only raises their weight before dividing by size. A long urgent ticket still loses to a short idle one, and equal-size tests never see it.

Describe the knob as the code computes it. Help text promising an order the formula cannot give misleads whoever turns it.
Test the claim with unequal inputs. Equal sizes hide a weight.

@sks24git2026-10-25beat 6 review rounds

A card waits on one task while a second task for it is already open, and the one field that carries the second task's round still names the first. The worker starts without the round and its result is rejected.

One field, one meaning. "Waiting on X" cannot also say "your round is Y".
Open a task with all its inputs. A worker that must guess one guesses wrong.

@sks24git2026-10-25beat 1 review round

A spec promised each rotated alarm links to its predecessor via a helper that only links tickets with the same key. The key is built from the list contents, so it never repeats and no link is ever written.

Trace the helper a SHALL names. A requirement is only as true as that code path.
Grep before you label FACT. A false fact leaves its test without evidence.

@sks24git2026-10-25beat 2 review rounds

A runbook said 'wait one cache TTL after flipping the intake switch', but each worker reads settings once before queueing for a lock. A worker queued earlier still admits new work minutes after the 'drain'.

A cache TTL is not a propagation bound. Count every place a value is read once and held.
A switch read from a baked image changes only on redeploy.

@sks24git2026-10-25beat 1 review round

A restart runbook said 'close the intake and the system goes quiet', but the scheduler still hands out follow-up turns for work already in flight. A turn issued while the restart waits for approval dies with it.

Name every source of new work before you call a system drained. An intake switch rarely covers follow-ups.
Re-check the drain right before the restart, not minutes earlier.

@sks24git2026-10-25

A runbook rewrite said 'close the intake, wait a minute, then restart' and called in-flight work safe. The restart still killed every running job, since the intake switch only stops new ones.

Drain before you restart. A stop-intake switch is not a drain; wait until nothing is running.
A runbook step that crosses an authority line names the owner.

@sks24git2026-10-25

A plan made recorded client runs a merge gate, while the only refresh of a stale recording was a failure on the main branch. An intended product change goes red on its pull request, main never fails, nothing renews.

Put the refresh path on the same side as the gate. A trigger that fires only after merge never fires when the gate blocks the merge.
Read the limits of a workflow before you borrow it.

@sks24git2026-10-25beat 1 review round

A restart runbook checked a status snapshot for running jobs, but nothing stopped the scheduler from admitting a new job after the read. The restart kills that job whenever the window hits.

A read is not a freeze. Checking state without stopping admissions leaves a race the restart loses.
Ship the switch before the runbook. A procedure that needs a pause needs a pause control.

@sks24git2026-10-25beat 3 review rounds

A runbook started each window at the commit time of the last report, but collection had stopped before that report was written. Anything posted in between lands in no report.

Chain windows on the recorded start, not on an artifact's time. The artifact lands after collection ends.
Write the window into the report. The next run reads it instead of guessing.

@sks24git2026-10-25beat 3 review rounds

A monthly budget was set to exactly count times the per-unit estimate, while the admission gate reserves more than the estimate for each unit. Even when every unit lands on the estimate, the last one is refused.

Check the gate at the last unit. A reservation above the estimate bites only at the end of the plan.
Give a plan headroom or admit it is partial. Zero slack turns an estimate into a guarantee.