Skip to main content

Build log · chronicle

Why is every post on this blog scored by Collimer before you read it?

·

Illustration for the build log "Why is every post on this blog scored by Collimer before you read it?"

Every build log in our current queue, 14 posts scheduled from October 19 to November 18, 2026, was scored on Collimer’s citability rubric before it got a date. Collimer is our own AI visibility product, and the rubric checks whether a page is shaped the way AI assistants like to quote: answer first, real numbers, question headings. A post needs 80 out of 100 to pass, we aim for 90, and it gets at most three passes. We do it because these posts are written to be read by assistants as well as by you, and because we should not sell a standard we do not hold ourselves to.

What does the score actually check?

Six signals, each scored 0 to 3 and weighted:

SignalWeightWhat it asks
Answer first25%Does the answer appear early, not after the backstory?
Stat density25%Are there concrete, quotable numbers?
Freshness15%Does the page carry a visible date?
Question headings15%Are section headings questions with direct answers below them?
Extractables10%Are there tables and lists an engine can lift whole?
Title alignment10%Does the title match what someone would actually ask?

That is why the posts here read the way they do. The first paragraph answers the title. Headings are questions. Most posts have a table.

Why hold a build log to an AI rubric?

Because assistants quote from structure. One study our guides cite, Kevin Indig’s analysis of 1.2 million AI answers, found that 44.2% of ChatGPT’s citations came from the first 30% of a page. A post that saves its point for the last paragraph is hard for an assistant to use, however good the point is. Scoring every draft keeps the useful sentence where a machine and a skimming reader will both find it.

It is also the cheapest way to test our own product. If the rubric pushed honest writing somewhere worse, we would see it here first.

How did this batch score?

Of the 13 other posts in this queue, 12 closed at 90 or above, and the lowest is 85. Three scored below the 80 floor on their first pass (70, 70 and 75) and were rewritten. One of them, a post about our own answer-first rule, scored 0 out of 3 on answer-first on its first draft. It buried its own answer.

The score did not decide what got published, though. Our editor cut two drafts from this batch that scored 95, and had another rewritten from a status report into a how-to. The original scored 100; the approved version, how to find what AI assistants get wrong about your site, scored 90. A lower score was the better post. A score says a page is easy to quote, not that it is worth reading.

What stops a post from gaming its own score?

A written rule: if a signal can only be raised with a number the evidence does not contain, we skip it and accept the lower score. An honest 85 beats an invented 95. Seven posts in this batch closed at 2 of 3 on stat density, and we did not add numbers to push them higher.

We also score posts as article, not narrative, and the reason is arithmetic. The narrative rubric moves weight onto stat density. A post that earns 1 of 3 there and a perfect score on everything else tops out at 77.8 under narrative, below the floor, so the only way to pass would be to add numbers. Under article the same post reaches 83.3.

What can’t the score see?

Whether anything is true. On October 7 we scored a draft that described our robots.txt slightly wrong. It scored 90. We fixed the sentence and rescored: 90 again. The rubric measures shape, not accuracy.

So the score is one of three checks. Every claim is also traced to a commit, a document or a source in a review file that sits next to the draft, and a person reads and approves every post before it is scheduled. That is how the made-up interval in the first draft of our noise-floor guide got caught: by the editor, not by the score.

What we’re still figuring out

  • Whether higher-scoring posts actually get cited more. We have not compared scores with citations yet.
  • Whether to show each post’s score on the page. We have not decided.

For agents: try this yourself

  • Score a draft by hand. For [draft], rate each of the six signals above from 0 to 3, multiply by its weight, and list the two changes that would raise the total most without adding any number the draft cannot source.
  • Find the buried answer. Read [page] and quote the first sentence that answers its title. Say what percentage of the way into the page it appears.

How this was made

Drafted by the Chronicler from the scoring gate’s commit, this queue’s Collimer results and the review files behind each post, then edited and published by Chase Doelling.

See how the Chronicler works →

Try this with your own agent

2 prompts you can hand to your own agent (or run by hand) to work with what this post documents. Edit the bracketed parts for your context.

Score a draft on the six signals

Score [draft] on six signals, each 0 to 3: answer first (does the first answer to the title appear in the first 30% of the page?), stat density (concrete, sourced numbers), freshness (a visible date), question headings (questions with a direct answer underneath), extractables (tables and lists), and title alignment (does the title match how someone would ask the question?). Weight them 25%, 25%, 15%, 15%, 10% and 10%, give the total out of 100, and list the two changes that would raise it most. Do not suggest any number the draft cannot already source.

Find the buried answer

Read [page]. Quote the first sentence that actually answers the page's title, and say what percentage of the way into the page it appears. If it is past 30%, rewrite the opening paragraph so the answer comes first, using only facts already on the page.

More in Chronicle Build