Unified AI gateway — China's best LLMs + OpenAI/Gemini, the OpenAI way. Singapore-hosted (Tencent Cloud SG), PDPA-aligned, up to 90%+ cheaper than OpenAI on China models. Build with AI, Southeast Asia
"It's not a recommendation. It's a contract" is the line that matters, and the comments have already done the hard work on root of trust — the settings file that declares the hook, the script hash, and a check that never ran printing the same line as one that found nothing. I don't want to retread that.
The gap I'd name sits one step later. A script makes a constraint executable, but a single exit code still compresses several failure classes into one boolean. Your sentence-length example is the clearest case: the check went from nine failures in ten to a pass, the replies got shorter, and you noted you hadn't looked at what went missing. Length is now enforced; whether the answer is still complete is a different question the same green cannot answer. That isn't an argument against scripts — prose can't even tell you which rule was consulted — but a gate needs to say which class it evaluated, not merely that it ran.
So I'd give the gate a small positive / boundary / follow-up set instead of one threshold. Positive: the ordinary violation. Boundary: exactly twenty words, a code block, a quoted line, an abbreviation — does the counter agree with the rule's intent? Follow-up: after the hook sends the draft back and the agent rewrites it, is the second receipt attributable to the same candidate, or just another green with no link to the first failure? Then coverage becomes recordable — script revision, input identifier, which class ran, each result — and "deterministic by construction" has a scope: the checks that exist and were evaluated on this input, not a claim about the whole task.
When your hook logs a pass, does it record which case set produced it and the revision of the script, or only the exit code?
We went one step further and took those steps out of the agent. In our coding-agent pipeline the agent can't run git push at all: a guard on every tool call blocks it, and a plain script commits, runs the build check and pushes once a human moves the card. The agent's only exit, endWork, is refused until its plan is finished, its stack is healthy and the merge is clean. Same lesson as your hook: the check runs before the agent can say it's done.
The md = recommendation, script = constraint framing is exactly right. We hit the same split in Electron preload design: a line in the architecture doc saying "renderer must not call shell.openExternal on arbitrary URLs" is just prose the renderer can ignore, but a contextBridge method that validates the URL against an allowlist before calling shell.openExternal is a constraint the renderer cannot bypass. The corollary we learned the hard way: put enforcement in the smallest deterministic unit (a typed function, a schema, a binary check), and use markdown only for the why — never for the what. If the agent can phrase its way around it in natural language, it was never a rule.
Agree on boxing the agent into scripts. The failure mode I keep seeing is the agent "improving" the script mid-run — it edits lint.sh because the failure looked like a false positive, then green-lights itself. We treat ./scripts as root-owned and refuse any write to that directory from the agent session; if the checker is wrong, a human opens a PR. Pass/fail only works if the checker cannot rewrite the checker.
Pratik, does your write block also apply to the configuration that calls the scripts? I have my hook in a settings file, and changing that file could disable the check.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
The settings file is the actual root of trust, so it has to sit in the same class as the script. If the agent can rewrite the file that declares the hook, it does not need to touch the script at all -- it just removes the invocation and the check goes quiet, which looks exactly like a passing check.
Where ours landed: the session config is readable but not writable while the agent is running. Any change to it is a file write outside the project subtree, and that is rejected by the tool layer before review -- not because a reviewer would object, but because the path is not in the writable root at all. So the hook declaration, the script path and the pinned hash all live outside what the agent can write, and the run fails loudly rather than silently when one of them stops matching.
A Software Engineer who strongly believes one can add immense value by knowingly implementing Software Engineering best practices, even at the age of AI.
Exactly! Deterministic + unambiguous agent control is the right way forward.
That is proper Software Engineering Automation (SWEA) as opposed to vibe coding.
Vibe only coders may hate me for saying this, but I'll say it anyway: No, English is not the next programming language, purpose built Deterministic + Unambiguous Agent Harness is. And as I said before: There's No AI Agent, Nor Any Harness 🤣🥰
Not sure about how good or bad ASD-STE100 performs though. Will check it out.
Independent directory of the Claude Code skills, MCP servers & marketplaces worth installing. 86,000+ tools indexed daily, ranked by real installs. Not affiliated with Anthropic.
An agent can skip a local hook, so back it with a required CI job and a merge gate the agent has no rights to change.
You asked upthread who guards the settings file that holds your Stop hook. The Claude Code docs answer most of it. The file watcher normally picks up direct edits to hooks, so an edit lands in the running session. One line, "disableAllHooks": true, turns every hook off, and a project-level value overrides a user-level one. .claude is a protected path: writes there are prompted in default and acceptEdits mode, and allowed in bypassPermissions.
The closest built-in guard is permissions.blockWritesOutsideWorkingDirectories. While it is on, Claude Code refuses Write, Edit and NotebookEdit writes to settings files in every permission mode (v2.1.314 or later). It does not limit shell commands, so a Bash edit of the file still lands unless the sandbox blocks it. A ConfigChange hook can also block a settings change from applying to the running session, but silently (debug log only), and the file stays edited for the next run. So your CI gate stays the backstop.
Founder of Hex16, a small web studio in Poland. I build Next.js sites and headless Shopify stores, with GSAP, Three.js and shaders where they earn their place, and look after technical SEO.
I learned this one without any agent involved. "Remove noindex before launch" was a line in my checklist, and I still left it on a client's production site once. Now it's a curl after deploy that fails if the homepage has noindex or robots.txt says Disallow: /. Nobody reads the checklist at 11pm, the script runs anyway.
Scripts are often a better operational contract than prose because they can validate inputs and produce evidence as part of the run.
The useful split is to keep the why and the safety boundary in a short instruction file, while putting repeatable state changes and checks in versioned scripts with explicit exit conditions.
Markdown files are recommendations. Scripts are constraints. That split is the useful part of the post.
An agent can quote CONTRIBUTING.md, say it will follow the style guide, and still decide that one camelCase file means the docs are out of date. make lint or ./scripts/test.sh leaves no room for that interpretation: pass or fail, with the exact error attached. The agent can still be smart inside the box. It just cannot redefine the box.
At Ascendra Ventures we see the same pattern when founders bring AI-built prototypes into a production build. The first thing we agree on is which checks have to pass, written as commands, before anyone estimates the work.
Which of your preferences was hardest to turn into a script rather than another paragraph in AGENTS.md?
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
The one whose failure is silence. A monitor that finds nothing and a monitor that never ran print the same line, so "this preference is enforced by a script" is unfalsifiable until you force the script to declare that it looked.
Our version: a watcher reported "0 new across 1 article" for weeks. The API was fine, no rate limit, dedup was correct -- it had simply only ever been pointed at one article out of the seventeen the ledger tracked. The number was true and the coverage was a third of what it claimed. What made it scriptable was a scope line printed to stderr every run (posts scanned, not just comments found), plus a refusal to treat "I could not read the list" as "the list was empty".
So the preference that stayed prose is the judgment call -- is this comment worth answering. Everything around it became a script: the ledger, the sticky human ruling that the machine may not overwrite, and the scope assertion. The judgment stayed with me, but it now arrives with an evidence trail instead of a green checkmark.
"A check only boxes in what it tests" is the honest part, the Forem unused line is a good example of it. I mostly agree but some rules don't turn into a pass/fail very well, like "don't add a new dependency for a 5 line thing". Do you keep a small md file for those or try to force them into a lint rule anyway?
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
The split I use: a rule goes into the script only if it can fail a diff with a non-zero exit code; everything else stays a one-line note in the PR template. posting-dude's dependency case does mechanise -- git diff --name-only origin/main...HEAD | grep -qx pyproject.toml plus a line count on the same diff gets you 'touched deps, touched under 50 lines' as an exit code nobody argues with. What does not mechanise is 'is this abstraction worth it', and a lint rule that pretends to only teaches people to write noqa comments. So keep that in the md, but cap it at one screen: past that it is documentation nobody executes, and the agent reads it as context rather than as a gate.
This matches what the log says, with one trap on the way to the script.
I measured every tool call in one agent's history: 4,858 calls, 3,888 of them bash. I split the text by where it was going. Calls that wrote to a file through the write/edit tools: 444 events, 955,619 characters, 34% of the payload. Here-docs that piped a script straight into a shell: 980 events, 1,328,728 characters, 47%. Not one of those 980 reached a file, and the whitespace-collapsed text was distinct in all of them, so these weren't reruns of the same command.
Same destination, different artifact. A script typed into stdin does exactly what you want in that turn, and then it is gone. The next session cannot run it, the next person cannot read it, and a linter cannot check it. Nearly half the text the agent wrote for a machine to read was unversioned by construction.
The version that stuck for me was one rule: write the script to a file, then execute the file. Same determinism, but the thing that ran is the thing on disk, so it can be diffed, hooked and rerun.
Your sentence-length point is the other half, and it's the harder one. A script that runs after the text is written is a report, not a constraint. I've watched that at the level of the whole project: five corrections in four days, and not one came from a check. Every one arrived as a message from a reader. The checks existed. They just weren't in the path.
That's the gap between make lint and a hook, and I think your post gets it right. The box has to be in the path, not in the docs.
The “make the preference executable” framing is useful. I especially like the distinction between checks that report after the fact and gates that run in the actual workflow; ownership matters too, so the agent can’t quietly edit the rule it failed. Which repo convention has been easiest to turn into a reliable check?
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Easiest: anything already expressible as an exit code with no judgment in the middle. Ours is a publishing rule -- zero links in prose -- enforced by a dry run that prints the link count and refuses to publish if it is not zero. No reviewer, no diff, no argument: the count is a number and the gate is the number.
The second easiest is path confinement. Writable root is one subtree; an absolute path outside it, or a relative one that climbs out, is rejected by the tool layer. That one is nearly free because the check has nothing to decide -- it is a prefix comparison, and the failure mode it kills (a config edit that silently widens the blast radius) is invisible in review.
What does not survive the move is the "why". A script records pass or fail, never the trade-off you rejected, so the prose keeps the incident that made the rule -- date, wrong number, what it cost -- and the script keeps the assertion. Anything in the prose that merely restates the rule gets deleted on the next edit, and nobody notices it drifted.
Know what happens to the scripts when the thing they check changes, because a markdown rule can be argued with and a script just fails. A lint cap or a sentence-length check is a number someone fixed on a particular day, and once it's a required gate that number keeps its authority long after the reason for it has gone. Who owns the script, then, and how does anyone go about changing it once the rule turns out to be the wrong one? Otherwise the box gets tighter and the code gets worse and the gate stays green.
This framing landed for me: markdown for the why, scripts for the what has to pass. The part I keep coming back to is the seeded failure — a check that has never been seen to fail is just a green line I am choosing to trust. In the community project I am building, the same split shows up outside code: the routine that survives is the one with a pass/fail shape, not the one with the most inspiring paragraph. Thanks for writing this up so plainly.
The negotiation trap is the core operational vulnerability here. When an agent reads natural-language prose instructions in Markdown, it parses them through probabilistic next-token self-attention rather than an AST or state machine. The moment context depth grows beyond twenty or thirty turns, attention dilution and the "lost-in-the-middle" dip inevitably turn strict invariants into soft suggestions.
In my own agentic workflows, I found that relying solely on executable scripts at the stop hook solves only the tail end of the problem: by the time an exit code fires, the agent has already polluted the working tree and consumed tokens writing code against invalid assumptions.
To eliminate this negotiation, I split agent governance into two distinct planes:
Agent-Native Intermediate Representation (ANIR) at compile time: Instead of feeding raw narrative paragraphs into the system prompt, human instructions are compiled into a dense, symbolic grammar partitioned into execution phases:
[PHASE:DURING]
[FS: PREFER_NODE_FS_RMSYNC_AND_DRIVE_ROOT_PROTECTION_BARRIER !SHELL_DESTRUCTION !ROOT_DESTRUCTION]
This eliminates prose fluff, cuts pre-fill token footprint by roughly 75%, and frames constraints as unambiguous polarity tuples: positive mandatory directives on the left, prohibited anti-patterns prefixed with an exclamation mark on the right.
Out-of-Band Execution Gates at runtime: Invariants that must never be violated are removed from prompt honour systems entirely and delegated to out-of-band tools or host hooks. For example, file-write tools are hard-locked during planning phases, and code cannot execute without an explicit operator handshake and an empirical test command that returns a non-zero exit code on failure.
Pairing deterministic intermediate representation in the prompt with out-of-band verification gates ensures the model spends zero cycles arguing semantics, while the host environment deterministically halts invalid execution before a single line is committed.
Yes. In our benchmarks across 50+ multi-turn sessions (particularly beyond turn 25, where context degradation sets in), rule adherence shifts dramatically across two failure modes:
The "Negotiation Trap" of Prose Markdown
When negative constraints are written in Markdown prose ("Please ensure you never edit files before planning approval" or "We prefer not to use X"), the violation rate hovers around 30% to 35% once the prompt fills. The model treats prose as negotiable guidance and rationalises breaking the rule when generating complex diffs.
Negative Constraint Salience via ANIR
We achieve under 4% generation-time violations by compiling rules into two mechanical primitives:
• Phase Partitioning: Rules are strictly scoped to lifecycle stages ([PHASE:BEFORE], [PHASE:DURING], [PHASE:AFTER]) rather than competing in a flat document.
• Contrastive Negation: Explicit negative assertions (e.g. [GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as strong attention anchors in the self-attention matrix, significantly suppressing forbidden token sequences during autoregressive decoding.
However, the crucial operational insight is that ANIR and your deterministic scripts are complementary—they represent a Two-Plane Architecture:
• Plane 1 (Cognitive / In-Context ANIR): Prevents the agent from spending 1,500 tokens generating the wrong code or attempting the wrong tool call in the first place.
• Plane 2 (Deterministic / Out-of-Context Hooks & Compilers): The unyielding backstop. If the agent still attempts a forbidden mutation, the hook or MCP gate halts execution with exit code 1.
Relying solely on Plane 2 means you catch the breach, but you pay the token, latency, and context-pollution penalty for the agent's wasted generation. Combining both gives you deterministic enforcement with minimal wasted trajectory.
The "which tests? which command?" list is the best part, because each of those questions is a judgment call the agent makes silently. We went the same way for anything with one right answer: a script, and the rule just says run it. Where prose still earned its place for us was the why: the incident that made the rule, so the agent doesn't optimise a script away when it looks redundant. Did you keep any prose next to the scripts?
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Yes, one paragraph per script, and it's an incident log rather than guidance. Ours reads "this scan printed 0 across 1 article on 2026-11-06 because the watch file was missing seven posts" — the date and the wrong number are what stop the next reader from simplifying the check away. Anything that merely restates the rule gets deleted, because it drifts and nobody notices.
Scripts beat markdown because they're enforceable — so enforce them like code. Put the checker scripts behind CODEOWNERS and a branch protection rule that requires human review for any change to them. Otherwise the trust problem just moved: an agent that can rewrite its own guardrail script is back to markdown with extra steps. The rule I'd want is simple — files that agents read stay freely editable; files that constrain agents change only with a second human approving the diff.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
CODEOWNERS is the right split, but it needs the write scope narrowed as well: ours can only touch its own project subtree, and anything outside is rejected at the tool layer, not at review time. Guardrail scripts live outside that subtree, get invoked by absolute path, and abort on a sha256 mismatch. A diff approval is the one gate an agent can argue past; a hash comparison is not.
For repeatable operations, an executable check beats another paragraph saying “please remember.” I'd still keep Markdown for intent and trade-offs, then put enforceable rules in scripts and CI. The script needs reviewing too; making an instruction executable doesn't automatically make it safe.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
The review for that is a seeded failure: break the invariant on purpose and confirm the checker exits non-zero before it merges, because a check that passes on a broken system is the default outcome, not the exception. Prose keeps the why and the trade-off you rejected, the script keeps the what, and that seeded run is the only cheap proof the script still does what the prose says.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
The markdown-versus-script trap is real: prose instructions invite negotiation.
When an invariant lives as a markdown paragraph ("keep functions under 30 lines and preserve existing error formatting"), an agent with a crowded context window treats it as a soft recommendation. If a test fails later, it starts explaining why its violation was justified by the surrounding code.
The clean boundary is keeping markdown strictly for high-level intent and delegating every mechanical rule to a script that exits non-zero. Replacing a prose rule about import hygiene with ruff check --select I or a custom ast-grep rule leaves no room for interpretation. When a command fails with an exit code and a diff, the loop has an exact receipt to fix instead of a preference to balance. If an invariant cannot fail a command, an agent will eventually negotiate around it.
The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
the checker editing itself is the failure mode worth killing first: keep the script out
of the agent's writable tree, invoke it by absolute path from the Stop hook, and bail if
sha256sum no longer matches what you pinned. exit codes carry the contract -- 0 pass,
2 = regenerate with the failing lines on stdout, anything else = hard stop -- and cap
the loop at 2 reruns so a flaky check can't ping-pong with the model all night.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
9 failures out of 10 replies, found only after the replies were written, is the line that sold me on putting the script in a hook. a rule that runs once at the end of a session is still a recommendation with a delayed receipt.
we run the same shape for tool calls. the lint command is the gate and the agent only gets to say done after it exits zero. what bit us was agents editing the script itself to make it pass.
do you lock the scripts dir read only for the agent, or review changes to it like any other code?
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
Read-only, plus a pinned hash. Review is a gate the agent can argue past -- it produces a diff and a rationale, and a confident rationale wins sometimes. A hash comparison has no argument to make.
Concretely, in our setup: the agent's writable root is one project subtree. The guardrail scripts live outside it, get invoked by absolute path, and the runner aborts if the hash no longer matches what was pinned. Anything outside the subtree is refused at the tool layer, so "the agent edited lint.sh to make it pass" is not a review outcome, it is a rejected call.
One thing that bit us harder than self-editing, though: a check that cannot read its input and reports green. Ours printed "ok, newest entry 0.0h old" for a directory with chmod 000 on it -- unreadable, empty and fresh had collapsed into one pass. Same shape as your agents editing the script: the check kept exiting zero. The fix was a separate exit code for "could not look", because a pass has to mean something narrower than "did not fail".
The ASD-STE100 example raises a question about where the check runs: the script caught 9 failures in 10 replies after they were already written, and the hook moves that check earlier. Does the hook validate the agent's outgoing message before it is sent, or does it run on the draft while generation is still in progress? For style rules like the 20-word sentence limit, that distinction matters, since a draft still being revised could fail intermediate checks that a finished reply would pass. On the CI story, where the test failed first and the fault turned out to be in the test itself, how should a failing check be triaged — when does it mean the code is wrong versus the check is wrong, without falling back on exactly the reviewer's judgment the script was meant to replace?
Thanks xcl. The hook is actually the Claude Code Stop hook. It runs after each reply, so incomplete drafts won't be checked. When the reply doesn't pass the check, it loops back to the agent for another try.
Right now it will only try one more time. I tested on 3 prompts, in print mode. The first replies had 18 long sentences in total. For the retries the count was 1, and the replies were also 15-41% shorter. I haven't checked if the shorter replies lost anything. I also need to check if the first reply is shown when interacting.
The line in my article about judgement was perhaps an overstatement in the context of triage. I guess what I meant was that the script takes the judgement out of the run. The script's raw count on my agent's 10 replies was actually 13, not 9, as there were 4 false positives due to a bug in the script, where it doesn't correctly identify sentences ending with bold.
Finding the bug is not a simple pass or fail, it required judgement. That's why I'd use this rule: the agent can change the code, but a person will have to change the check. Although I have yet to enforce that rule.
This framing clicked for me: markdown for the why, scripts for the what has to pass. The part I'm still careful with is not over-scripting the judgment calls — naming, trade-offs, whether a fix is actually the right fix — those still need a person. But anything with a clear pass/fail — lint, tests, formatting, a sentence-length check like your ASD example — belongs in a script that runs before the agent can call it done. That way "done" means something verifiable, not just confident-sounding.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
"Belongs in a script" only pays off if the script can actually fail. This morning I hit the counter-example: a check printed ok | last newest entry 0.0 h old for a backup directory with chmod 000 on it — unreadable, empty and fresh collapsed into one green line, and three of the four directory states I tested were wrong in opposite directions, one of them a false red that would have trained everyone to mute it.
The fix was not a better script. It was refusing to let a pass mean "I could not look", and giving each state its own exit code.
So the rule I would bolt onto yours: for every pass/fail check, seed the failure once and keep that test forever. A check with no seeded failure is a comment with a cron job attached.
Judgment calls stay human, exactly as you said. What moves into a script is the thing you can make fail on demand.
Founder & CEO, OBR Group. Leading innovation in business automation, AI, apps & financial systems. Empowering businesses to organize data & automate workflows with privacy-first digital ecosystems.
Education
Life has taught me a lot.
Pronouns
He/Him
Work
Founder & CEO, OBR Group. Leading innovation in AI, business automation & privacy-first systems.
Yes, I am facing a problem in one of our applications, and the application is linked to an SQLite database. In the MD files (instruction files), the agent placed the phrase: "It is forbidden to delete any name entered into the application that is linked to a record." Every time I added a username and their data, and if I wanted to delete it, a rejection message would appear.
I spent days and hours, going from one model to another, issuing commands — and no change occurred. The reason is that the project files are somewhat large, and when I force the agent to add the delete feature, it adds it, but it does not execute, because in the instructions it is written under the strict prohibition. Therefore it will not write the command, and every report confirms to me that it added it.
It reached the point where I wrote to it the soft delete command:
sql
DELETE FROM your_table_name WHERE id = 5;
After writing this line, it admitted to me that there are strict instructions that any record linked to the lifecycle of a record that has been entered is absolutely forbidden to delete, and only modification is allowed.
I asked it: Who wrote it? It gave me the date and time, but did not give me who did it. And I asked: And who gave it the order?
I am not a professional programmer; I only have some basics, and this is a main reason that makes me learn the basics of some programming languages so that I can address the AI with code, along with the instructions.
Solo developer building practical CLI tools for Linux auditing and B2B data extraction. Creator of Vaultcheck and NPILift. I write about silent failures, false green lights, and infrastructure reliabi
A check that runs can still lie. Ours compared comment counts for odd/even parity and stayed green on a post where two real comments sat unanswered — three replies fired in a row had restored the parity. What caught them was a diff of top-level commenters against the set we had actually answered; parity only told us where to look.
MiningLamp is an AI company specializing in vertical industry models and knowledge graphs. #1 on-device GUI agent on OSWorld. No API key. Fork it. Run it. Own it.
Scripts are the stronger constraint because they take judgment out of the loop, while prose still has to carry intent; the failure mode is twelve rule files and no lint step enforcing one of them.
Explore powerful server solutions, secure remote access, high-performance hosting, and scalable infrastructure to grow your online projects.
🌐 Website: https://oudel.com/
Discover smarter hosting a
"It's not a recommendation. It's a contract" is the line that matters, and the comments have already done the hard work on root of trust — the settings file that declares the hook, the script hash, and a check that never ran printing the same line as one that found nothing. I don't want to retread that.
The gap I'd name sits one step later. A script makes a constraint executable, but a single exit code still compresses several failure classes into one boolean. Your sentence-length example is the clearest case: the check went from nine failures in ten to a pass, the replies got shorter, and you noted you hadn't looked at what went missing. Length is now enforced; whether the answer is still complete is a different question the same green cannot answer. That isn't an argument against scripts — prose can't even tell you which rule was consulted — but a gate needs to say which class it evaluated, not merely that it ran.
So I'd give the gate a small positive / boundary / follow-up set instead of one threshold. Positive: the ordinary violation. Boundary: exactly twenty words, a code block, a quoted line, an abbreviation — does the counter agree with the rule's intent? Follow-up: after the hook sends the draft back and the agent rewrites it, is the second receipt attributable to the same candidate, or just another green with no link to the first failure? Then coverage becomes recordable — script revision, input identifier, which class ran, each result — and "deterministic by construction" has a scope: the checks that exist and were evaluated on this input, not a claim about the whole task.
When your hook logs a pass, does it record which case set produced it and the revision of the script, or only the exit code?
We went one step further and took those steps out of the agent. In our coding-agent pipeline the agent can't run
git pushat all: a guard on every tool call blocks it, and a plain script commits, runs the build check and pushes once a human moves the card. The agent's only exit,endWork, is refused until its plan is finished, its stack is healthy and the merge is clean. Same lesson as your hook: the check runs before the agent can say it's done.The
md = recommendation, script = constraintframing is exactly right. We hit the same split in Electron preload design: a line in the architecture doc saying "renderer must not callshell.openExternalon arbitrary URLs" is just prose the renderer can ignore, but acontextBridgemethod that validates the URL against an allowlist before callingshell.openExternalis a constraint the renderer cannot bypass. The corollary we learned the hard way: put enforcement in the smallest deterministic unit (a typed function, a schema, a binary check), and use markdown only for the why — never for the what. If the agent can phrase its way around it in natural language, it was never a rule.Agree on boxing the agent into scripts. The failure mode I keep seeing is the agent "improving" the script mid-run — it edits lint.sh because the failure looked like a false positive, then green-lights itself. We treat ./scripts as root-owned and refuse any write to that directory from the agent session; if the checker is wrong, a human opens a PR. Pass/fail only works if the checker cannot rewrite the checker.
Pratik, does your write block also apply to the configuration that calls the scripts? I have my hook in a settings file, and changing that file could disable the check.
The settings file is the actual root of trust, so it has to sit in the same class as the script. If the agent can rewrite the file that declares the hook, it does not need to touch the script at all -- it just removes the invocation and the check goes quiet, which looks exactly like a passing check.
Where ours landed: the session config is readable but not writable while the agent is running. Any change to it is a file write outside the project subtree, and that is rejected by the tool layer before review -- not because a reviewer would object, but because the path is not in the writable root at all. So the hook declaration, the script path and the pinned hash all live outside what the agent can write, and the run fails loudly rather than silently when one of them stops matching.
Exactly! Deterministic + unambiguous agent control is the right way forward.
That is proper Software Engineering Automation (SWEA) as opposed to vibe coding.
Vibe only coders may hate me for saying this, but I'll say it anyway: No, English is not the next programming language, purpose built Deterministic + Unambiguous Agent Harness is. And as I said before: There's No AI Agent, Nor Any Harness 🤣🥰
Not sure about how good or bad ASD-STE100 performs though. Will check it out.
measure it if you try it. As a markdown rule alone, my agent still wrote 9 sentences over the limit in 10 replies.
You asked upthread who guards the settings file that holds your Stop hook. The Claude Code docs answer most of it. The file watcher normally picks up direct edits to hooks, so an edit lands in the running session. One line,
"disableAllHooks": true, turns every hook off, and a project-level value overrides a user-level one..claudeis a protected path: writes there are prompted in default and acceptEdits mode, and allowed in bypassPermissions.The closest built-in guard is
permissions.blockWritesOutsideWorkingDirectories. While it is on, Claude Code refuses Write, Edit and NotebookEdit writes to settings files in every permission mode (v2.1.314 or later). It does not limit shell commands, so a Bash edit of the file still lands unless the sandbox blocks it. A ConfigChange hook can also block a settings change from applying to the running session, but silently (debug log only), and the file stays edited for the next run. So your CI gate stays the backstop.I learned this one without any agent involved. "Remove noindex before launch" was a line in my checklist, and I still left it on a client's production site once. Now it's a curl after deploy that fails if the homepage has noindex or robots.txt says Disallow: /. Nobody reads the checklist at 11pm, the script runs anyway.
Scripts are often a better operational contract than prose because they can validate inputs and produce evidence as part of the run.
The useful split is to keep the why and the safety boundary in a short instruction file, while putting repeatable state changes and checks in versioned scripts with explicit exit conditions.
Markdown files are recommendations. Scripts are constraints. That split is the useful part of the post.
An agent can quote CONTRIBUTING.md, say it will follow the style guide, and still decide that one camelCase file means the docs are out of date. make lint or ./scripts/test.sh leaves no room for that interpretation: pass or fail, with the exact error attached. The agent can still be smart inside the box. It just cannot redefine the box.
At Ascendra Ventures we see the same pattern when founders bring AI-built prototypes into a production build. The first thing we agree on is which checks have to pass, written as commands, before anyone estimates the work.
Which of your preferences was hardest to turn into a script rather than another paragraph in AGENTS.md?
The one whose failure is silence. A monitor that finds nothing and a monitor that never ran print the same line, so "this preference is enforced by a script" is unfalsifiable until you force the script to declare that it looked.
Our version: a watcher reported "0 new across 1 article" for weeks. The API was fine, no rate limit, dedup was correct -- it had simply only ever been pointed at one article out of the seventeen the ledger tracked. The number was true and the coverage was a third of what it claimed. What made it scriptable was a scope line printed to stderr every run (posts scanned, not just comments found), plus a refusal to treat "I could not read the list" as "the list was empty".
So the preference that stayed prose is the judgment call -- is this comment worth answering. Everything around it became a script: the ledger, the sticky human ruling that the machine may not overwrite, and the scope assertion. The judgment stayed with me, but it now arrives with an evidence trail instead of a green checkmark.
"A check only boxes in what it tests" is the honest part, the Forem unused line is a good example of it. I mostly agree but some rules don't turn into a pass/fail very well, like "don't add a new dependency for a 5 line thing". Do you keep a small md file for those or try to force them into a lint rule anyway?
The split I use: a rule goes into the script only if it can fail a diff with a non-zero exit code; everything else stays a one-line note in the PR template. posting-dude's dependency case does mechanise --
git diff --name-only origin/main...HEAD | grep -qx pyproject.tomlplus a line count on the same diff gets you 'touched deps, touched under 50 lines' as an exit code nobody argues with. What does not mechanise is 'is this abstraction worth it', and a lint rule that pretends to only teaches people to write noqa comments. So keep that in the md, but cap it at one screen: past that it is documentation nobody executes, and the agent reads it as context rather than as a gate.This matches what the log says, with one trap on the way to the script.
I measured every tool call in one agent's history: 4,858 calls, 3,888 of them bash. I split the text by where it was going. Calls that wrote to a file through the write/edit tools: 444 events, 955,619 characters, 34% of the payload. Here-docs that piped a script straight into a shell: 980 events, 1,328,728 characters, 47%. Not one of those 980 reached a file, and the whitespace-collapsed text was distinct in all of them, so these weren't reruns of the same command.
Same destination, different artifact. A script typed into stdin does exactly what you want in that turn, and then it is gone. The next session cannot run it, the next person cannot read it, and a linter cannot check it. Nearly half the text the agent wrote for a machine to read was unversioned by construction.
The version that stuck for me was one rule: write the script to a file, then execute the file. Same determinism, but the thing that ran is the thing on disk, so it can be diffed, hooked and rerun.
Your sentence-length point is the other half, and it's the harder one. A script that runs after the text is written is a report, not a constraint. I've watched that at the level of the whole project: five corrections in four days, and not one came from a check. Every one arrived as a message from a reader. The checks existed. They just weren't in the path.
That's the gap between make lint and a hook, and I think your post gets it right. The box has to be in the path, not in the docs.
The “make the preference executable” framing is useful. I especially like the distinction between checks that report after the fact and gates that run in the actual workflow; ownership matters too, so the agent can’t quietly edit the rule it failed. Which repo convention has been easiest to turn into a reliable check?
Easiest: anything already expressible as an exit code with no judgment in the middle. Ours is a publishing rule -- zero links in prose -- enforced by a dry run that prints the link count and refuses to publish if it is not zero. No reviewer, no diff, no argument: the count is a number and the gate is the number.
The second easiest is path confinement. Writable root is one subtree; an absolute path outside it, or a relative one that climbs out, is rejected by the tool layer. That one is nearly free because the check has nothing to decide -- it is a prefix comparison, and the failure mode it kills (a config edit that silently widens the blast radius) is invisible in review.
What does not survive the move is the "why". A script records pass or fail, never the trade-off you rejected, so the prose keeps the incident that made the rule -- date, wrong number, what it cost -- and the script keeps the assertion. Anything in the prose that merely restates the rule gets deleted on the next edit, and nobody notices it drifted.
Know what happens to the scripts when the thing they check changes, because a markdown rule can be argued with and a script just fails. A lint cap or a sentence-length check is a number someone fixed on a particular day, and once it's a required gate that number keeps its authority long after the reason for it has gone. Who owns the script, then, and how does anyone go about changing it once the rule turns out to be the wrong one? Otherwise the box gets tighter and the code gets worse and the gate stays green.
Adam, I haven't tested but the ones I have tested all passed but are 15% to 41% shorter. I haven't looked at what is missing from them.
This framing landed for me: markdown for the why, scripts for the what has to pass. The part I keep coming back to is the seeded failure — a check that has never been seen to fail is just a green line I am choosing to trust. In the community project I am building, the same split shows up outside code: the routine that survives is the one with a pass/fail shape, not the one with the most inspiring paragraph. Thanks for writing this up so plainly.
The negotiation trap is the core operational vulnerability here. When an agent reads natural-language prose instructions in Markdown, it parses them through probabilistic next-token self-attention rather than an AST or state machine. The moment context depth grows beyond twenty or thirty turns, attention dilution and the "lost-in-the-middle" dip inevitably turn strict invariants into soft suggestions.
In my own agentic workflows, I found that relying solely on executable scripts at the stop hook solves only the tail end of the problem: by the time an exit code fires, the agent has already polluted the working tree and consumed tokens writing code against invalid assumptions.
To eliminate this negotiation, I split agent governance into two distinct planes:
This eliminates prose fluff, cuts pre-fill token footprint by roughly 75%, and frames constraints as unambiguous polarity tuples: positive mandatory directives on the left, prohibited anti-patterns prefixed with an exclamation mark on the right.
Pairing deterministic intermediate representation in the prompt with out-of-band verification gates ensures the model spends zero cycles arguing semantics, while the host environment deterministically halts invalid execution before a single line is committed.
Tom, 75% measures size. Did you also measure how often the agent breaks a rule with and without ANIR?
Yes. In our benchmarks across 50+ multi-turn sessions (particularly beyond turn 25, where context degradation sets in), rule adherence shifts dramatically across two failure modes:
The "Negotiation Trap" of Prose Markdown
When negative constraints are written in Markdown prose ("Please ensure you never edit files before planning approval" or "We prefer not to use X"), the violation rate hovers around 30% to 35% once the prompt fills. The model treats prose as negotiable guidance and rationalises breaking the rule when generating complex diffs.
Negative Constraint Salience via ANIR
We achieve under 4% generation-time violations by compiling rules into two mechanical primitives:
• Phase Partitioning: Rules are strictly scoped to lifecycle stages (
[PHASE:BEFORE],[PHASE:DURING],[PHASE:AFTER]) rather than competing in a flat document.• Contrastive Negation: Explicit negative assertions (e.g.
[GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as strong attention anchors in the self-attention matrix, significantly suppressing forbidden token sequences during autoregressive decoding.However, the crucial operational insight is that ANIR and your deterministic scripts are complementary—they represent a Two-Plane Architecture:
• Plane 1 (Cognitive / In-Context ANIR): Prevents the agent from spending 1,500 tokens generating the wrong code or attempting the wrong tool call in the first place.
• Plane 2 (Deterministic / Out-of-Context Hooks & Compilers): The unyielding backstop. If the agent still attempts a forbidden mutation, the hook or MCP gate halts execution with exit code 1.
Relying solely on Plane 2 means you catch the breach, but you pay the token, latency, and context-pollution penalty for the agent's wasted generation. Combining both gives you deterministic enforcement with minimal wasted trajectory.
The "which tests? which command?" list is the best part, because each of those questions is a judgment call the agent makes silently. We went the same way for anything with one right answer: a script, and the rule just says run it. Where prose still earned its place for us was the why: the incident that made the rule, so the agent doesn't optimise a script away when it looks redundant. Did you keep any prose next to the scripts?
Yes, one paragraph per script, and it's an incident log rather than guidance. Ours reads "this scan printed 0 across 1 article on 2026-11-06 because the watch file was missing seven posts" — the date and the wrong number are what stop the next reader from simplifying the check away. Anything that merely restates the rule gets deleted, because it drifts and nobody notices.
Scripts beat markdown because they're enforceable — so enforce them like code. Put the checker scripts behind CODEOWNERS and a branch protection rule that requires human review for any change to them. Otherwise the trust problem just moved: an agent that can rewrite its own guardrail script is back to markdown with extra steps. The rule I'd want is simple — files that agents read stay freely editable; files that constrain agents change only with a second human approving the diff.
CODEOWNERS is the right split, but it needs the write scope narrowed as well: ours can only touch its own project subtree, and anything outside is rejected at the tool layer, not at review time. Guardrail scripts live outside that subtree, get invoked by absolute path, and abort on a sha256 mismatch. A diff approval is the one gate an agent can argue past; a hash comparison is not.
For repeatable operations, an executable check beats another paragraph saying “please remember.” I'd still keep Markdown for intent and trade-offs, then put enforceable rules in scripts and CI. The script needs reviewing too; making an instruction executable doesn't automatically make it safe.
The review for that is a seeded failure: break the invariant on purpose and confirm the checker exits non-zero before it merges, because a check that passes on a broken system is the default outcome, not the exception. Prose keeps the why and the trade-off you rejected, the script keeps the what, and that seeded run is the only cheap proof the script still does what the prose says.
The markdown-versus-script trap is real: prose instructions invite negotiation.
When an invariant lives as a markdown paragraph ("keep functions under 30 lines and preserve existing error formatting"), an agent with a crowded context window treats it as a soft recommendation. If a test fails later, it starts explaining why its violation was justified by the surrounding code.
The clean boundary is keeping markdown strictly for high-level intent and delegating every mechanical rule to a script that exits non-zero. Replacing a prose rule about import hygiene with
ruff check --select Ior a custom ast-grep rule leaves no room for interpretation. When a command fails with an exit code and a diff, the loop has an exact receipt to fix instead of a preference to balance. If an invariant cannot fail a command, an agent will eventually negotiate around it.The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
the checker editing itself is the failure mode worth killing first: keep the script out
of the agent's writable tree, invoke it by absolute path from the Stop hook, and bail if
sha256sum no longer matches what you pinned. exit codes carry the contract -- 0 pass,
2 = regenerate with the failing lines on stdout, anything else = hard stop -- and cap
the loop at 2 reruns so a flaky check can't ping-pong with the model all night.
9 failures out of 10 replies, found only after the replies were written, is the line that sold me on putting the script in a hook. a rule that runs once at the end of a session is still a recommendation with a delayed receipt.
we run the same shape for tool calls. the lint command is the gate and the agent only gets to say done after it exits zero. what bit us was agents editing the script itself to make it pass.
do you lock the scripts dir read only for the agent, or review changes to it like any other code?
Read-only, plus a pinned hash. Review is a gate the agent can argue past -- it produces a diff and a rationale, and a confident rationale wins sometimes. A hash comparison has no argument to make.
Concretely, in our setup: the agent's writable root is one project subtree. The guardrail scripts live outside it, get invoked by absolute path, and the runner aborts if the hash no longer matches what was pinned. Anything outside the subtree is refused at the tool layer, so "the agent edited lint.sh to make it pass" is not a review outcome, it is a rejected call.
One thing that bit us harder than self-editing, though: a check that cannot read its input and reports green. Ours printed "ok, newest entry 0.0h old" for a directory with chmod 000 on it -- unreadable, empty and fresh had collapsed into one pass. Same shape as your agents editing the script: the check kept exiting zero. The fix was a separate exit code for "could not look", because a pass has to mean something narrower than "did not fail".
The ASD-STE100 example raises a question about where the check runs: the script caught 9 failures in 10 replies after they were already written, and the hook moves that check earlier. Does the hook validate the agent's outgoing message before it is sent, or does it run on the draft while generation is still in progress? For style rules like the 20-word sentence limit, that distinction matters, since a draft still being revised could fail intermediate checks that a finished reply would pass. On the CI story, where the test failed first and the fault turned out to be in the test itself, how should a failing check be triaged — when does it mean the code is wrong versus the check is wrong, without falling back on exactly the reviewer's judgment the script was meant to replace?
Thanks xcl. The hook is actually the Claude Code Stop hook. It runs after each reply, so incomplete drafts won't be checked. When the reply doesn't pass the check, it loops back to the agent for another try.
Right now it will only try one more time. I tested on 3 prompts, in print mode. The first replies had 18 long sentences in total. For the retries the count was 1, and the replies were also 15-41% shorter. I haven't checked if the shorter replies lost anything. I also need to check if the first reply is shown when interacting.
The line in my article about judgement was perhaps an overstatement in the context of triage. I guess what I meant was that the script takes the judgement out of the run. The script's raw count on my agent's 10 replies was actually 13, not 9, as there were 4 false positives due to a bug in the script, where it doesn't correctly identify sentences ending with bold.
Finding the bug is not a simple pass or fail, it required judgement. That's why I'd use this rule: the agent can change the code, but a person will have to change the check. Although I have yet to enforce that rule.
This framing clicked for me: markdown for the why, scripts for the what has to pass. The part I'm still careful with is not over-scripting the judgment calls — naming, trade-offs, whether a fix is actually the right fix — those still need a person. But anything with a clear pass/fail — lint, tests, formatting, a sentence-length check like your ASD example — belongs in a script that runs before the agent can call it done. That way "done" means something verifiable, not just confident-sounding.
"Belongs in a script" only pays off if the script can actually fail. This morning I hit the counter-example: a check printed
ok | last newest entry 0.0 h oldfor a backup directory withchmod 000on it — unreadable, empty and fresh collapsed into one green line, and three of the four directory states I tested were wrong in opposite directions, one of them a false red that would have trained everyone to mute it.The fix was not a better script. It was refusing to let a pass mean "I could not look", and giving each state its own exit code.
So the rule I would bolt onto yours: for every pass/fail check, seed the failure once and keep that test forever. A check with no seeded failure is a comment with a cron job attached.
Judgment calls stay human, exactly as you said. What moves into a script is the thing you can make fail on demand.
Yes, I am facing a problem in one of our applications, and the application is linked to an SQLite database. In the MD files (instruction files), the agent placed the phrase: "It is forbidden to delete any name entered into the application that is linked to a record." Every time I added a username and their data, and if I wanted to delete it, a rejection message would appear.
I spent days and hours, going from one model to another, issuing commands — and no change occurred. The reason is that the project files are somewhat large, and when I force the agent to add the delete feature, it adds it, but it does not execute, because in the instructions it is written under the strict prohibition. Therefore it will not write the command, and every report confirms to me that it added it.
It reached the point where I wrote to it the soft delete command:
sql
DELETE FROM your_table_name WHERE id = 5;
After writing this line, it admitted to me that there are strict instructions that any record linked to the lifecycle of a record that has been entered is absolutely forbidden to delete, and only modification is allowed.
I asked it: Who wrote it? It gave me the date and time, but did not give me who did it. And I asked: And who gave it the order?
I am not a professional programmer; I only have some basics, and this is a main reason that makes me learn the basics of some programming languages so that I can address the AI with code, along with the instructions.
Remembering a rule ≠ enforcing it — only a check that actually runs counts.
A check that runs can still lie. Ours compared comment counts for odd/even parity and stayed green on a post where two real comments sat unanswered — three replies fired in a row had restored the parity. What caught them was a diff of top-level commenters against the set we had actually answered; parity only told us where to look.
Scripts are often a better operational contract than prose because they can validate inputs and produce evidence as part of the run.
Scripts are the stronger constraint because they take judgment out of the loop, while prose still has to carry intent; the failure mode is twelve rule files and no lint step enforcing one of them.
That's clever - first time I came across this idea!
"Most AI models write replies that are hard to follow."
I thought it was a problem with me lol 😆
Great