Every number in a technical article should come from somewhere. @xbill shows how to prove it: research-kit (a Claude Code port of my Antigravity Research Kit) audits the numbers, and publishing-kit blocks the publish until every disputed number is fixed.
This publish gate's whole value turns on how "disputed" gets defined. Is it the model flagging anything it can't trace to a source, or a person having to sign off before it clears? What happens to a figure that's disputed but right, because that's the decisive test for whether people trust the gate or work around it.
Spot on @nark3d That’s the exact test of whether a gate gets trusted or bypassed.
In xbill’s setup, research-kit writes every quantitative claim into a CLAIMS.md ledger backed by an evidence/ directory, and publishing-kit treats any unresolved disputed entry like a failing build.
So "disputed" isn't just the model guessing it's a claim whose number doesn't match the primary source or recomputed code output. And if a figure is disputed but actually right (e.g., a local benchmark or an unindexed source), you don't bypass the gate you attach the evidence (code output, log, or explicit provenance note) to CLAIMS.md so the ledger clears with an audit trail.
The goal isn't letting the model overrule the author; it's making sure no number ships unexamined.
We're a place where coders share, stay up-to-date and grow their careers.
Top comments (2)
This publish gate's whole value turns on how "disputed" gets defined. Is it the model flagging anything it can't trace to a source, or a person having to sign off before it clears? What happens to a figure that's disputed but right, because that's the decisive test for whether people trust the gate or work around it.
Spot on @nark3d That’s the exact test of whether a gate gets trusted or bypassed.
In xbill’s setup, research-kit writes every quantitative claim into a CLAIMS.md ledger backed by an evidence/ directory, and publishing-kit treats any unresolved disputed entry like a failing build.
So "disputed" isn't just the model guessing it's a claim whose number doesn't match the primary source or recomputed code output. And if a figure is disputed but actually right (e.g., a local benchmark or an unindexed source), you don't bypass the gate you attach the evidence (code output, log, or explicit provenance note) to CLAIMS.md so the ledger clears with an audit trail.
The goal isn't letting the model overrule the author; it's making sure no number ships unexamined.