An operating scorecard that runs itself

Our operating scorecard used to work exactly as well as we remembered to open it. Its gates lived in a file we updated by hand, on a schedule that really meant whenever someone thought to look. That is not automation. It is a diary with extra columns.

This is the next chapter of the thread that started with treating the studio’s own build-out as a series of tracked bets and continued with giving the marketing side an operator scorecard that refuses to fabricate a trend. This week the internal scorecard got the same treatment: a real database, a schedule it doesn’t need reminding about, and a place it pushes itself to.

What we shipped

Three commits over three days took the gates off manual entry. The first pulled them straight from a Neon-backed database instead of a hand-maintained file, and added a live clock plus Today, attention, and experiments surfaces on top of the existing scorecard page. The second wired the tick to run from the lookout job that was already scheduled for Monday mornings, so the numbers update themselves instead of waiting on someone to run a script. The third pushed the rendered card to Center Pane after every tick, so the version in the hub and the version on the studio’s own page stop drifting apart between manual syncs.

A fourth commit added a fact logger, npm run log, built to take under 15 seconds so a raw number gets written down before it’s forgotten instead of reconstructed from memory a week later.

What broke (or got harder than expected)

The friction wasn’t a bug. It was not knowing where the truth actually lived: whether a gate’s real state was tracked in this repo or in Center Pane, when both seemed to claim it. Tracing both systems side by side settled it, but it’s the kind of question that should have had an obvious one-line answer and didn’t. Nothing here technically failed, but a system where two places can each plausibly hold the current state is a small design defect on its own.

A decision worth naming

The clearest test of whether any of this was worth building showed up almost immediately, and it wasn’t a metric anyone had asked for. A batch of scans turned up from a region the studio has no customers in, the kind of thing a hand-updated spreadsheet would never have surfaced, because nobody looks that closely at a column of raw numbers. The instinct was to wave it off: probably a bot, a scanner, background noise, not a big risk. The decision that mattered was refusing that instinct and turning it into backlog tickets instead of a shrug.

Building BI infrastructure for a one-to-two-person studio can look like solving a problem that doesn’t exist yet. This is the counter-argument, one week in: the pipeline caught something worth a real security question before there was enough data, by most people’s definition, to justify having built it at all.

What’s next

• Work the anomalous-traffic tickets the tick opened, rather than let the “probably nothing” read close them without a look.
• Watch the Monday tick run unattended for a few more weeks before treating three days of runtime as proof it holds up on its own.

What we’re still figuring out

We don’t know yet whether the odd batch of scans was a bot, a monitoring service, or something that deserves closer attention. The tickets exist because we don’t know, not because we’ve already decided. And we don’t know whether the 15-second fact logger changes behavior or just adds a command nobody reaches for once a real deadline is in the room. Both answers come from watching what actually happens over the next few Monday ticks, not from this one.

---

For agents: try this yourself

If you’re building any kind of business dashboard, adapt one of these. Full prompt text lives in this post’s prompts sidecar, operating-scorecard-that-runs-itself.prompts.md.

• Reproduce the source-of-truth trace. Pick one metric your team tracks in more than one place. Trace which system is actually authoritative when they disagree, and write that down somewhere visible.
• Apply the anomaly-to-ticket habit. The next time a number in your own metrics looks slightly off, resist closing the question with “probably nothing.” Open a ticket instead, even a one-line one, before deciding it isn’t worth a look.
• Critique your own automation’s blind spot. For any dashboard you update automatically, ask what an anomaly would actually look like in it, and whether you’d notice one odd row out of thousands.

---

How this was made

Drafted by the Chronicler from the build sessions behind this work, then edited and published by Brian Wones.

See how the Chronicler works →

I track [METRIC OR GATE] in more than one place: [SYSTEM A] and [SYSTEM B]. Compare what each one currently says. If they disagree, trace which one is actually authoritative when a human has to decide, and write a one-line note in the repo saying which system wins and why.

Look at [DASHBOARD OR METRICS SOURCE] for anything that looks slightly off in the last week, an unusual spike, an unfamiliar source, a number that doesn't match the usual pattern. Instead of telling me it's probably nothing, open a short backlog ticket describing exactly what looks odd and why it's worth a second look.

I have a dashboard or report that updates automatically from [DATA SOURCE]. Tell me what an anomaly in that data would actually look like once it reaches the automated view, and whether a single unusual row out of thousands would be visible to me or would get averaged away.