Limit Before Filter

What goes wrong

A row cap (LIMIT, slice, page size) is applied before the filter, dedup or merge that decides which rows qualify. Unqualified rows use up the cap, qualified rows are cut off, and the 'has more' flag is wrong.

Example

Fetch the first 10 rows from the database, then filter in code: 3 remain, though 20 matching rows exist and the page says 'no more'.

Not to be confused with

The order of limit and filter is the defect — not a wrong label on what was counted (Aggregate measures the wrong population) and not a partial total presented as complete (Partial measurement claims complete coverage).

Discovered by

@sks24git GitHub

Field notes · 11

@sks24git2026-11-05beat 1 review round

A batch read takes the oldest twenty open rows, but skips the ones it cannot settle without closing them. Twenty stuck rows then fill every batch, and every newer row waits forever behind them.

Select only rows the loop will move. A row that stays put after the pass holds its slot in every later batch.

@sks24git2026-11-04beat 1 review round

A cleanup job takes the newest N candidates first and only then checks which ones need fixing. Healthy items that are picked again every run fill the batch, and the broken ones behind them are never reached.

Filter before you limit. A batch cap applied to unfiltered rows starves the rows that matter.
Catch each item's error. One bad row must not stop the whole pass every time.

@sks24git2026-11-04beat 1 review round

A read budget refreshed expired cache rows first, ahead of items with no row at all. On a cold cache each pass re-read the top of the list and the rest stayed unmeasured.

Fill gaps before you refresh. Old data is a fallback; a gap has none.
Test the outage path of a guard. A limit that holds only when the cache is up fails when needed.

@sks24git2026-11-03beat 2 review rounds

A wait-for-more gate counts every ready item before a later step drops the stale ones, which carry old start times. One stale item opens the gate early, so work ships in small lots and the wait does nothing.

Count only what will really ship. Run the gate after every filter that can drop an item.
Test with mixed ages. When all items share one age, min and max look the same.

@sks24git2026-11-01beat 3 review rounds

A cleanup pass takes the oldest rows in a fixed-size batch and then skips the ones it must keep. Once the kept rows fill the batch, every pass reads the same rows and the ones behind them are never cleaned.

Filter before you limit. Skipping rows after a capped read can starve everything behind them.
Promise the bound the code enforces. A published deadline must survive the slow path.

@sks24git2026-10-31beat 5 review rounds

A batch for tomorrow is cut from the first N items before some of them leave today on their own. The rest of the list is never added back, so tomorrow sends a short batch and items slip a day.

Fill a batch after you remove what already left. Slicing first and filtering second leaves holes nobody sees.

@sks24git2026-10-28beat 2 review rounds

A daily cap was applied while picking items, before the sender dropped ones already sent and not yet due. Day two re-picks the same batch, the sender drops it all, and the rest of the queue never ships.

Apply quotas after exclusions, not before. A cap on the wrong side of a filter starves everything behind it.
Test the no-answer day. Carryover only shows when nobody replies.

@sks24git2026-10-25

A list helper cut each source to its first thirty items and never said it had cut anything. A member who could read only items past the cut saw an empty list with no note, as if nothing were installed.

Return the cut with the data. A cap the caller cannot see turns "truncated" into "empty".
Filter after you page, not before. Capping before the access check hides exactly what passes it.

@sks24git2026-10-13beat 2 review rounds

A picker looks at only the first page of open pull requests and drops the rest. Projects whose fitting change sits further down are never chosen.

Page through every list you filter. A first page is a sample, not the set.

@sks24git2026-09-12beat 3 review rounds

Two jobs cut their output by the same operator setting; only one of them bounded how many pieces that could make. Lower the setting and a single export becomes thousands of files, each one re-signed on every poll.

When you copy half a mechanism, list the bounds the original carried. A limit on one form of an operation and not the other is a gap no test of either form sees.

@sks24git2026-09-12beat 2 review rounds

A resumable job validated its checkpoint by row and part counts but re-derived the part size each run from live config. Lower the config mid-job: the next run re-cuts smaller and appends, rows repeat, counters say done.

Validate a checkpoint by every input that shaped it. Counts agree across two different cuts.
Freeze live config into the job at enqueue. A value re-read per run is a second writer of the plan.