Two fixes that were quietly guessing, corrected the same week

A fix that passes its own test can still be wrong. Two unrelated systems proved that the same week: a database-pool resize made to close an incident quietly reproduced the exact problem it was supposed to prevent, and a marketing-attribution default had been guessing “web” for every conversion it couldn’t otherwise explain.

How do you verify a bug fix actually fixed the bug?

Check the fix’s own stated reasoning against the code, not against the incident report that closed it. Both fixes here were believed to be resolved. Neither one was visibly broken. That is what made them dangerous: a system that crashes gets noticed immediately, but a system that compiles, passes review, and quietly keeps doing the wrong thing can run that way for a long time.

The pool resize that reproduced its own incident

An earlier incident, connection-pool exhaustion, got a pool-sizing fix whose stated rationale was that background workers didn’t query the affected repository. That rationale was never checked against which modules actually import it. Seventeen worker modules query the repository the fix assumed was safe; zero query the one it assumed was catching the load. The fix relocated the exhaustion instead of preventing it. The correction sizes the pool for the workers that actually use it, and corrects a second stale claim along the way: a connection pooler believed to be multiplexing connections in production was never in the path at all.

The attribution default that always said “web”

Separately, new-user and new-scan attribution silently fell back to “web” on every path where the true source couldn’t be determined, including the Stripe webhook worker. That meant every paid conversion whose real channel wasn’t recognized got counted as web traffic by default. A funnel report built on that data could never show web underperforming, because every gap in the data was quietly backfilled with the one channel that could never look bad. The fix records “unknown” instead of guessing, across users, organizations, and scans.

Why these two belong in one post

Pairing a database-perf fix with a marketing-attribution fix risks reading as two minor bugs bolted together for volume, and that risk is real enough to name directly. What justifies the pairing is the shared root cause: both were previously “fixed” to look resolved, a pool resize that passed its own test and a fallback that always returned a value, without anyone checking whether the fix’s own reasoning was actually true. It is the same shape as a scorecard you have to remember to update not actually being a scorecard, and the same shape as three mechanisms reporting success while producing nothing. Three systems, three different failure shapes, one identical root cause: success was defined as “the fix compiled” instead of “the fix’s own claim is actually true.”

What we’re still figuring out

Both fixes closed the specific instance we found. We have not gone back through every other closed incident asking the same question, whether its fix’s stated rationale was ever actually checked against the code. That audit would be the honest next step, and it hasn’t happened yet.

---

For agents: try this yourself

If you maintain a system with closed incidents or default-fallback values, adapt one of these. The full prompt text lives in this post’s prompts sidecar, two-fixes-that-were-quietly-guessing.prompts.md.

• Reproduce the rationale check. Pick your last three closed incidents. For each fix, write down the specific claim its rationale depends on, then check that claim against the current code, not the incident write-up.
• Critique your own silent defaults. Find every place your system falls back to a default value when the real value is unknown. Check whether that default can ever make a report look better than reality, and if so, replace it with an explicit “unknown” instead.

---

How this was made

Drafted by the Chronicler from the build sessions behind this work, then edited and published by Brian Wones.

See how the Chronicler works →

List your last 3 closed incidents or bug fixes. For each one, write down the specific claim its fix's rationale depends on being true (e.g. "workers don't query this repo", "this pool isn't under load"). Check each claim directly against the current code, not against the incident write-up or PR description. Report any where the claim doesn't hold.

Find every place in this codebase where a value silently falls back to a default when the real value can't be determined (attribution sources, categorizations, statuses). For each one, check whether that default could make a report or metric look better than reality if the true value were known. Where it could, replace the default with an explicit "unknown" value instead.