DEV Community

Robina Mirbahar
Robina Mirbahar

Posted on

Every number in a technical article should come from somewhere. @xbill shows how to prove it: research-kit (a Claude Code port of my Antigravity Research Kit) audits the numbers, and publishing-kit blocks the publish until every disputed number is fixed.

Sign in to view linked content

Top comments (2)

Collapse
 
nark3d profile image
Adam Lewis •

This publish gate's whole value turns on how "disputed" gets defined. Is it the model flagging anything it can't trace to a source, or a person having to sign off before it clears? What happens to a figure that's disputed but right, because that's the decisive test for whether people trust the gate or work around it.

Collapse
 
robinamirbahar profile image
Robina Mirbahar •

Spot on @nark3d That’s the exact test of whether a gate gets trusted or bypassed.

In xbill’s setup, research-kit writes every quantitative claim into a CLAIMS.md ledger backed by an evidence/ directory, and publishing-kit treats any unresolved disputed entry like a failing build.

So "disputed" isn't just the model guessing it's a claim whose number doesn't match the primary source or recomputed code output. And if a figure is disputed but actually right (e.g., a local benchmark or an unindexed source), you don't bypass the gate you attach the evidence (code output, log, or explicit provenance note) to CLAIMS.md so the ledger clears with an audit trail.

The goal isn't letting the model overrule the author; it's making sure no number ships unexamined.