Timeout Leaves Work Running

What goes wrong

A timeout wrapper gives up waiting and returns to the caller, but nothing stops the work it was waiting for. The abandoned work keeps running, holding resources and writing results after the caller has moved on.

Example

A request is raced against a 5-second timer; the timer wins and the caller returns an error, but the request keeps running and writes its result to the database a minute later.

Not to be confused with

The missing piece is stopping the work and waiting for it to finish; a timeout that is merely too short, or a cancellation signal caught as an ordinary error (Swallowed Cancellation), are different bugs.

Discovered by

@sks24git GitHub

Field notes · 4

@sks24git2026-11-05

An abandoned background wait learns its run is over only when cleanup finishes, after the error handler's own slow reads. In that gap the orphan still writes its file into a box that is being torn down.

Mark work abandoned the moment its deadline fires. Later cleanup leaves the orphan a window.
Check that mark right before each write. An earlier check misses the awaits in between.

@sks24git2026-11-01beat 1 review round

A cleanup job got a stale timeout: past ten minutes a stuck run no longer blocks a new one, but nothing cancels it. A lasting hang adds one stuck run, and one held connection, every ten minutes until the pool runs dry.

Never replace a hung task without cancelling it. The old one keeps its resources.
Keep the backstop off the main loop's lock. One hang must not silence both.

@sks24git2026-10-30beat 3 review rounds

A watchdog sent one polite stop signal to a hung worker and then recorded the run as a clean finish. If the worker ignores the signal, it runs on to its deadline and the record still says done.

Follow a polite stop with a hard one after a short wait. A hung process may never handle the first.
Record the exit you saw, not the one you hoped for. A forced success hides a real failure.

@sks24git2026-09-11beat 2 review rounds

A page read 'agent must send files now' straight off a request column that only the request's own cleanup ever cleared. A worker killed mid-wait left it set, so a finished page kept demanding files for six hours.

Derive flags from the state machine, never from a raw column. The state orders terminal words first for this reason.
Give every waiting state its own current step. A generic fallback lies.