Test Relies On Real Timing

What goes wrong

A test decides pass or fail by real elapsed time — sleep 100 ms then assert, 'the callback must fire within 50 ms', a race between two timers — instead of waiting for the actual event. On a busy CI machine the scheduler shifts which timer wins, so correct code fails: a flaky test.

Example

The test sleeps 100 ms and then asserts the callback was called; it passes locally and fails on a loaded build machine where the callback fires at 130 ms.

Not to be confused with

Excludes real production timeout bugs, assertions about fixed time boundaries under a controlled clock, tests interfering through shared state (Test Runs Delete Each Other's Data), and screenshot tests sensitive to font rendering.

Discovered by

@sks24git GitHub

Field notes · 9

@sks24git2026-11-06beat 1 review round

A browser test read the page on a fixed clock that started before the page loaded, while the code under test starts its own timer only once its script runs. On a slow machine the test fails on a correct build.

Start a test clock where the code starts its clock. Page load time is not part of the timer under test.
Wait for a state, not a fixed sleep. A busy runner moves every fixed sleep.

@sks24git2026-11-05beat 1 review round

A self-check proved a wait was scoped by asserting that a 6-second animation was still running after a 4-second wait. On a slow machine that gap closes, and a correct wait fails the required check.

Prove a negative with a gap far past any limit. A margin of seconds fails under load.
Assert on a count you return, not on the clock. Counts don't drift.

@sks24git2026-11-04

A UI test slept a fixed time that sat just above the code's own timer, tied together by a comment only. On a slow runner the timer fires late, the check looks too early, and it grades nothing yet passes.

Wait for the state, not for a number of milliseconds. Fixed sleeps race the code they test.
Share timing constants between code and test. A comment is not a link.

@sks24git2026-11-04

A test took a share of a deadline and asserted the exact result, though the clock moved between setting the deadline and reading it. Two milliseconds of load make the value one short, and CI fails for nothing.

Assert a range on anything read from a moving clock. Exact values only hold on an idle machine.

@sks24git2026-11-04

A test for a timer race sent its three steps as separate round trips to the browser. On a slow machine the timer fired before the last step arrived, so the test passed with the fix removed.

Run a timing-race repro in one tick of the page. Round trips add delay you do not control.
Check a race test goes red without the fix, under load.

@sks24git2026-11-03

A slow-query fix was guarded by a wall-clock test on a tiny fixture where the old query was also fast. Put the slow query back and the test stays green, yet on a busy runner it fails for nothing.

Guard speed by shape, not by stopwatch. Count the queries or check the plan; a time limit on small data catches neither side.

@sks24git2026-11-01beat 1 review round

A test counts live server sessions once, right after closing a connection, by a name every test shares. Session exit is async and neighbours share the name, so the check fails at random and turns the build red.

Poll for async teardown, never read once. The server ends sessions after the client returns.
Tag test resources uniquely. Shared names let neighbours shift your count.

@sks24git2026-11-01beat 3 review rounds

A check trusted one file only if it was newer than another, and the tests wrote both a moment apart. Where the clock gives both writes one tick, the check reads nothing and the tests pass for the wrong reason.

Set file ages on purpose in tests that compare them. Two writes in a row can share a timestamp.

@sks24git2026-09-12

A new gate check could only see the defect at one screen width, and nothing held that width in its list. Delete it and the check stays green forever while the bug it was written for walks back in.

Pin the inputs, not just the judge. A perfect check proves nothing if its caller stops feeding it.
Grade a threshold from both sides. One-sided, it passes when the oracle is widened.