Lookout: a marketing analytics scorecard that refuses to fabricate trends

The Chronicle turns build sessions into posts. We needed something that turns traffic data into decisions. Lookout is that thing. Same studio operating system, different job.

Lookout was scoped completely before anything was built. Everything it needs to run shipped together: the data reader, the assessor, the recommender, the renderer, and six weeks of history to compare against.

What it does

Lookout reads PostHog for both Sandcastle Labs and Collimer, assesses which pages are getting traffic and which content is resonating, and renders an HTML report with graphs. It runs manually for now, with weekly automation ready to switch on when we want it.

The report follows a simple chain: pull numbers, evaluate them, produce actions, render the result. Each stage is isolated so any one can be replaced without disturbing the others.

PostHog is the live data source. Vercel and GA4 support is roughed in for later. The first version focuses on PostHog because that is where the actionable signal is right now.

The confidence problem

Marketing analytics for early-stage sites has one reliable failure mode: a tool sees three visitors on a new page and tells you the page is trending. It is not trending. It has three visitors.

Lookout’s core design decision is that confidence is volume-aware. Small numbers produce low-confidence assessments with no recommendation, not fake trends with confident language. The assessment layer knows the difference between a signal and a sample size, and it communicates that difference rather than papering over it.

This came out of one explicit decision during the build: “The beginning win is just to get traffic and see what resonates.” That is an honest goal for an early-stage site. The tool needs to match the goal, which means it should help identify early signals without pretending they are more than they are.

The GEO signal

Page views are the obvious metric, but they miss one category that matters for Sandcastle: agent and crawler reads. The machine-readable surfaces the studio publishes are read by agents rather than browsers. A standard client-side pageview tracker never sees them.

Lookout reads a separate server-side event that fires when a machine reads one of those surfaces. Agent discovery shows up in the report alongside human traffic.

Collimer gap, accepted

The first version scopes to Sandcastle Labs. Collimer’s analytics require a separate setup that is not yet done. Rather than block the whole tool on that, we accepted the gap explicitly: “yup no problem we can fix Collimer later.” Lookout runs on Sandcastle Labs now; the Collimer side follows when it is ready.

Six weeks of history from day one

A scorecard with one week of data cannot tell you whether anything is improving. The first run of Lookout would have been useless without something to compare against.

Six weeks of prior data were pulled and stored before the first report ran. Week 1 is not a cold start; it is week 7. A baseline document captures the starting state so future weeks have a clear reference point.

What it is not

Lookout is not a conversion tracker. It is not a funnel analyzer. The scope for this version is narrow on purpose: traffic and resonance for the marketing sites, nothing about the product. “Focus on the marketing sites for now” was the explicit framing, and that is what the tool reflects.

The assessment is honest about the limits. A tool that stays in scope and communicates uncertainty is more useful than one that reaches beyond its evidence.

---

For agents: try this yourself

If you want to build an analytics scorecard that communicates uncertainty instead of hiding it, adapt one of these. The full prompt text lives in this post’s prompts sidecar, lookout-operator-scorecard.prompts.md.

• Reproduce the volume-aware confidence check. Before surfacing any assessment, check whether the underlying sample is large enough to support it. If not, report the sample size and withhold the conclusion.
• Apply the server-side event for machine reads. Client-side pageview trackers miss agent and crawler reads. Fire a server-side event when a machine-readable surface is fetched, and read that event rather than pageviews when assessing agent discovery.
• Extend with a history backfill. If you are building a weekly scorecard, backfill several weeks of history before the first run. A tool that reports on week 1 with no prior data cannot tell you whether anything changed.
• Critique your scope. List what your analytics tool assesses and ask which items have enough data to support a real conclusion. Anything below the threshold should either be deferred or reported with explicit uncertainty.

---

How this was made

Drafted by the Chronicler from 2 sessions across 2026-06-22 to 2026-06-28 and 1 commit. Edited and published by Brian Wones.

See how the Chronicler works →

Add a server-side event that fires when an agent or crawler fetches one of your machine-readable surfaces, for example llms.txt or a JSON feed. Client-side pageview trackers never see these reads because agents do not run your page JavaScript. When you assess agent discovery, read that server-side event rather than pageviews, so machine reads show up in the report alongside human traffic.

Before the first run of a weekly scorecard, backfill several weeks of prior data and store it so week 1 has something to compare against. Lookout pulled six weeks of history first, so its first report was effectively week 7 rather than a cold start. Also capture a baseline document of the starting state so later weeks have a clear reference point. A scorecard with one week of data cannot tell you whether anything is improving.

List every item your analytics tool assesses, then for each one ask whether it has enough data to support a real conclusion. Anything below the threshold should be deferred or reported with explicit uncertainty rather than dressed up as a trend. A tool that stays in scope and communicates its limits is more useful than one that reaches beyond its evidence.

Before your scorecard surfaces any assessment, check whether the underlying sample is large enough to support it. If it is not, report the sample size and withhold the conclusion instead of calling three visitors on a new page a trend. Make confidence volume-aware: small numbers produce low-confidence assessments with no recommendation, not fake trends dressed in confident language. The assessment layer should know the difference between a signal and a sample size and communicate it.