1. The argument is being filed under the wrong heading
Most debates about agent reliability get filed as model problems. The proposed fix is a better model, a lower temperature, a longer system prompt, a different framework, or more evals. All of those can help. None of them change where the decision actually lives.
Our position is that reliability in an agent system is not a property of the model. It is a property of the architecture around it: which decisions the model is allowed to make, which ones are made by ordinary code, and whether the system keeps enough evidence to tell the two apart afterwards.
That is a technical position, not a product claim. The phrase we use for it is deterministic delivery, and it is worth stating precisely, because it is easy to oversell.
2. What the position actually says
Deterministic delivery is controllability, auditability and fallbacks at the architecture level. It is explicitly not a claim that the model will always be right. A model reading an ambiguous email can still get it wrong inside a perfectly built backbone.
Three conditions have to hold before we'd use the phrase:
- Every decision node leaves a replayable, traceable record: inputs, outputs, latency, and the acceptance criterion it was supposed to meet.
- Rule logic is deterministic by construction — same input, same state transition, same audit entry — while semantic nodes stay probabilistic and are measured rather than assumed.
- A failure can be located to a node, not just observed as a wrong final answer.
If any of those is missing, adding tests or tuning prompts treats a symptom. The gap is architectural.
This also fixes a common misreading. "Deterministic" here does not describe the model's output. It describes the parts of the system that are not allowed to improvise.
3. The split, and where the other side is right
The alternative is a probabilistic black box: each step decides what happens next, state lives inside the prose, a retry is a model call that sounds like a retry, authorization is a line in a prompt asking nicely. When the output is wrong, you get a paragraph, not a location.
We'd argue that's the wrong default for production. But it is not wrong everywhere, and a position that cannot say when its opposite is fine is a preference, not a position. For a one-off script, a throwaway prototype, or internal exploration where a wrong answer costs nothing and nobody has to explain it later, chaining model calls is the cheaper and more honest choice. A deterministic backbone earns its keep when failures must be attributed, outputs reach external users, an audit trail matters, or more than one person has to maintain the system past the demo.
So the real disagreement is not about whether to use workflows, or how autonomous an agent should be. It is about a boundary: which decisions require judgement, and which decisions require a diff? Framed that way, the boundary is usually much narrower than the initial design assumed.
4. Three constraints, one property
The position above is not implemented by a single component. Three engineering constraints produce it, and deterministic delivery is the property that emerges when all three hold. They are not a feature list; each has a criterion you can check and a characteristic way of failing.
C1 — hybrid orchestration. A deterministic backbone owns state, retries, authorization, audit trails and rollback; a model is invoked only at nodes requiring semantic judgement.
Criterion: count the nodes that could be written as an if. If the answer is "almost none," there is no orchestration, only a chain of prompts with extra latency.
Failure mode: state hidden in generated text, so a retry silently multiplies the same call and nobody can reconstruct what ran.
C2 — structured prompt contract. A versionable spec of role, hard constraints, I/O contract and fallback rules. Not a persona — persona in English means role-play, and role-play is the part that does not survive contact with production. What matters is the constraint block: missing_context → ask for clarification, never infer; output must be a typed object; if a constraint cannot be satisfied, emit fallback_reason and stop.
Criterion: you can diff the constraint set between two deploys and assert on it without running the whole agent.
Failure mode: the rule exists in a system prompt, but nothing records whether it was ever consulted on this input.
C3 — white-box evaluation. Capture per-node inputs, outputs, latency and acceptance criteria into replayable test sets, instead of judging the final answer by feel. Three sets, kept separate: positive (the happy path with grounded context), boundary (empty retrieval, contradictory sources, out-of-scope questions, malformed tool responses), and follow-up (multi-turn drift and corrections).
Criterion: a failure from last week can be replayed as a single node, with its original input and criterion.
Failure mode: evaluation that scores only the final answer, which tells you something is wrong and nothing about where.
Note the ordering. C1 defines where decisions live, C2 makes the rules at those boundaries explicit and testable, C3 proves they fire. Skipping C1 and going straight to C3 is the most common mistake: you end up with a large test suite over a system whose nodes only exist in the model's output text.
5. What this looks like in one trace
Consider a support agent that tells a customer refunds apply to a product category no policy document covers. Nobody notices until a customer complains, because the only thing the system kept was the final answer. An illustrative trace of that node would look like this:
{
"step": "answer_policy_question",
"input": { "question": "...", "retrieved_chunks": 0 },
"constraint": "missing_context -> ask_for_clarification",
"constraint_evaluated": false,
"fallback_reason": null,
"output": { "answer": "Refunds apply to this category" },
"latency_ms": 1840
}
Read against the three constraints: C2's rule exists but was never evaluated; C1 has no record of the state transition that should have stopped the call; C3 has no boundary case asserting on retrieved_chunks = 0. All three failed at once, and the trace is the only reason we can say so. The model produced a plausible continuation — that is what a language model does with no grounded context. Blaming hallucination ends the investigation; the trace starts it.
One detail worth keeping: log the evaluation, not just the spec. Plenty of systems record the constraint and assume it ran. The gap between "we wrote the rule" and "the rule fired on this input" is where most of these incidents live.
6. Scope and limits
A position is only useful if it says what it does not cover.
- It does not fix semantic errors on genuinely ambiguous input, missing knowledge in the retrieval corpus, or requirements that were wrong to begin with. A perfect backbone around a bad question still produces a bad answer.
- It does not mean fewer model calls, cheaper inference, or no hallucinations. There is a cost consequence — explicit state makes retries visible and spend attributable to a step instead of to "the agent ran long" — but that is downstream of the architecture, not the goal.
- It does not remove the need for model quality. It changes what you can do when model quality is insufficient: add a boundary case, tighten a fallback, move a decision into code, and prove the change with a replay.
- It is not worth the overhead when failures are cheap, invisible and unattributable — the cases in §3 where the other side is right.
What it does buy is a short, boring postmortem. Failures become addressable, "add a boundary case" becomes a normal engineering response instead of "tune the prompt and hope," and the model can be changed later without re-litigating the whole architecture. That is what keeps an agent maintainable past the demo — not a cleverer chain of prompts.
7. What follows
If this position holds, the first thing to build is not an eval suite. It is the definition of where the nodes are and who owns them, because every artefact above — the contract, the trace, the test sets — is indexed by that boundary. Get the boundary wrong and the rest becomes bookkeeping over a system you still cannot inspect.
Two questions follow, and we'd ask them before discussing models at all: where is the line between rule logic and model judgement written down so it can be diffed, and can a wrong answer be replayed to the exact node with its input, its criterion and the fallback that did not fire? In your stack today, does that boundary live in the orchestrator's config — or in whatever the model happens to emit at runtime?
Top comments (13)
The reframing from "which model" to "which decisions require judgement vs. a diff" is the right cut, and it's one most teams resist because writing a prompt is cheaper than writing the deterministic path — so the boundary silently creeps toward the model over time. The part I'd underline is the replayable record per node. In multi-agent setups the thing that actually kills you isn't a wrong answer, it's a wrong answer you can't localize: several agents in a chain, each sounding confident, and no way to say which one invented the bad fact. Once every node logs its inputs, output, and the acceptance criterion it was supposed to meet, debugging stops being archaeology. One question — how do you keep that boundary from drifting back over time? We've found it takes an explicit, recurring review ("should this node really be a model call?"), because the default gravity is always toward handing more to the model.
Agree on the drift direction — writing a prompt is cheaper than writing the deterministic path, so the boundary almost never migrates back on its own. A calendar-based review is better than nothing, but I'd make it trigger on observable signals instead.
Two that seem useful: when a node's acceptance criterion stops being machine-checkable and turns into "a human reads the output and decides", the decision has already moved into the model, regardless of what the config says. And when a structured prompt contract accumulates conditional branches, fallback clauses and "if X, instead do Y" lines across versions, that's the rule layer being rewritten in prose. Both show up in the diff, so the review can be driven by the contract's version history and by the test sets rather than by a quarterly reminder.
The multi-agent point is the sharper one though: localization is what the per-node record buys you, and drift destroys it silently — the chain still produces confident answers, but the boundary between a judgement and a diff is no longer recoverable after the fact.
Is your review cadence fixed, or triggered by changes to the contract and the test sets?
"The gap between 'we wrote the rule' and 'the rule fired on this input' is where most of these incidents live." This is the single sharpest systems observation published on agent architecture this year.
Your illustrative trace exposes the fatal flaw of first-generation agent observability: recording a constraint in a system prompt and assuming the model evaluated it during next-token prediction.
To answer your two concluding questions—and to offer a production pattern that closes that exact evaluation gap:
1. Where the line is written down: Compiled Invariant Grammar (ANIR)
When constraints are written in conversational Markdown (
CLAUDE.md,.cursorrules), the boundary is permeable. The model treats prose as negotiable guidance, yielding a 30% to 35% violation rate once the context fills past turn 25.We make constraints versionable and diffable by compiling them into an Agent-Native Intermediate Representation (ANIR):
• Lifecycle Phase Partitioning: Rules are strictly partitioned by state (
[PHASE:BEFORE],[PHASE:DURING],[PHASE:AFTER]) rather than competing in a flat document.• Contrastive Negation Tokens: Explicit negative assertions (e.g.
[GATE: PLANNING_LOCK !EDIT_FILES_BEFORE_APPROVAL !BYPASS_CPLAN]) act as high-salience attention anchors in the self-attention matrix, reducing generative drift to under 4%.Because this is a declarative manifest committed to Git, you can
git diffthe constraint rules between two deploys and assert on their presence without running the model.2. Where the boundary lives: Capability-Based Tool Gating
The boundary cannot live in whatever the model emits at runtime, and it cannot rely on the model self-attesting that a constraint was checked.
We solve your
"constraint_evaluated": falseproblem through Capability-Based Tool Gating (Object Capabilities for Agents):Instead of granting the model ambient authority to call execution tools:
This closes the gap between "the rule was written" and "the rule was evaluated." You do not trust the model to report whether it checked policy; the runtime physically denies execution unless the policy gate issued a capability receipt.
3. The Multi-Turn Entropy Reality
Your trace models a discrete node, but production coding agents loop over 30 to 60 turns.
Here, deterministic delivery faces a second failure mode: Context Entropy. Even with a structured prompt contract (C2), attention over early system invariants degrades non-linearly as the context window fills with intermediate tool diffs and terminal output.
True deterministic delivery requires an out-of-band Lifecycle Health Watchdog: monitoring turn-depth proximity, predicting attention compaction boundaries, and enforcing deterministic re-anchoring before the model suffers attention amnesia.
The Takeaway
The fundamental mistake of first-generation agent tooling was treating prompts as the operating system.
Prompts are untrusted user-space inputs; deterministic middleware is the kernel. You do not ask the application space to enforce process isolation, and you do not ask an LLM's next-token predictor to enforce an authorisation boundary. True deterministic delivery only emerges when the model is restricted to semantic judgement, while an external runtime owns the capability tokens, the execution gates, and the audit trail.
The capability receipt is the part I'd underline: it moves "was the constraint evaluated" from a model self-report to a precondition of execution, and a tool that physically cannot be invoked without a signed gate result is a different kind of boundary from prose in a system prompt. That's a stronger answer than the trace alone.
Where I'd still separate the claims is between existence and evaluation. A diffable manifest proves what was deployed; a capability receipt proves the gate issued a token. Neither proves the gate evaluated the right condition on this input. If retrieved_chunks is zero but the gate reads a field an earlier step already populated, it issues a receipt for a condition that never held — and the trace shows a successful, signed, compliant call. That's the same gap moved one layer down: the check becomes deterministic, but its input can still be wrong. Which is why the article records constraint_evaluated as an event with its inputs, rather than inferring it from the presence of either the rule or the receipt.
I'd leave the percentages out of the argument. Without a stated measurement protocol — what counts as a violation, over how many turns, judged by whom — a violation rate isn't comparable across setups, and two approaches measured differently will produce any number you want. The architectural claim doesn't need them.
Does the gate log its own condition evaluation into the trace, or only whether the token was issued?
Fair push—the stale witness problem. If the gate merely inspects an in-memory pipeline variable like
step.chunks_countthat an earlier turn populated, you've just automated an illusion.To your question: the gate has to log the raw witness, not just the token.
The pattern that keeps this honest in practice is treating the gate as a pure function over content-addressed inputs rather than ambient pipeline state. The token doesn't simply assert
PASS; it cryptographically binds the content digests of the actual retrieved chunks to the execution parameters. If retrieval returned zero chunks, the witness payload in the receipt is literallyhash([]), which fails the execution precondition downstream.That still doesn't solve whether the retrieval query itself was semantically "good"—that is the classical oracle problem, and anyone claiming architecture eliminates that is selling magic. But it does mean that in a post-incident audit you can replay the exact predicate offline against the recorded witness and get byte-for-byte reproducibility.
And I take your point entirely on the percentages. Without a published benchmark harness, turn-depth definition, and reproducible fixture, throwing around violation ratios just reads like hand-waving. Appreciate the pushback—it sharpens the boundary considerably.
The content-addressed witness is a real step past an ambient pipeline variable: the gate now has to bind what it actually saw, and hash([]) failing the precondition downstream is a check that cannot be talked out of it. I also agree with where you left the oracle problem — whether the retrieval query was any good is a semantic question, and no amount of hashing settles it.
One edge I'd still separate: hash([]) is evidence of absence, not evidence of why the input was empty. Corpus has no such policy, the query string was built wrong, and a scope filter narrowing the result set to zero all produce the same digest. The predicate replays byte-for-byte in all three cases, so the audit proves the gate was honest and still cannot tell you what went wrong upstream. A signed empty set is a stronger failure than an unsigned one, but it is still one failure class with three causes.
The small addition is to content-address the query side alongside the chunks: query text, filters, corpus version, retriever revision. Then the empty witness carries a reproducible reason, and the boundary set can be split into three cases instead of asserting the same thing for all of them — otherwise a regression suite passes while the root cause stays unknown. I'd also keep the witness and the receipt written by different components. A hash computed and consumed inside one process is a self-report with better cryptography; the rule logic stays deterministic by construction, but the evidence source does not become independent.
None of this makes the query semantically better, and it adds fields to every run. It only changes "precondition failed" into "precondition failed for a recorded, replayable reason". If you deliberately corrupt the query string while keeping retrieval empty, is there a field in the witness that changes with it — and if not, how would a post-incident replay distinguish "nothing was found" from "we looked in the wrong place" ?
A node replay needs a clear distinction between supplied input and ambient state. The same request can take a different deterministic branch if the rule reads the current clock, a mutable configuration, or a database row that changed after the original run.
I would capture the exact state revision and any clock or external-read values alongside the acceptance criterion, then replay against those recorded values. The test should also prohibit live side effects during reconstruction. That makes the single-node replay criterion stronger than rerunning the current implementation with the same question and hoping its environment still matches.
This tightens the replay criterion more than the original framing did. "Same input" quietly expands from the prompt to the entire read surface of the node — and once you accept that, a rule reading the current clock, a config value or a database row is reading an input it never declared.
Two places where the capture tends to be incomplete. External reads are broader than clock and config: feature flags, the model and tool versions actually used, and the routing decision itself all change which branch a deterministic rule takes. If those aren't in the record, replaying against recorded values still reproduces the wrong run. And the prohibition on live side effects only holds if capture happens at the boundary — one direct read inside the rule code bypasses any harness that assumes callers inject state.
That also suggests replay is a property of the interface, not of the logging: the node has to ask a provider for its clock and revision so the harness can substitute recorded values. Otherwise "replay with recorded values" is a convention that a single refactor can break.
In your records, do model version and routing choice count as state, or do you keep them as separate version tags?
For the replay contract I am proposing, both belong in the captured envelope, with different roles. The immutable model and tool revisions identify what executed. A routing choice is an output when replaying the router, but an input when replaying the downstream node that received the request.
I would preserve that boundary explicitly. The router test should recompute its choice from recorded inputs and compare it with the recorded choice; the downstream test should receive that recorded choice without consulting the live router. Merely attaching version tags would not enforce either behavior.
The output-of-the-router / input-of-the-downstream-node split is the sharper version of what I wrote, and it exposes what my earlier reply left implicit: one captured value has to be used in two incompatible roles depending on which node is under test. Version tags attach metadata; they don't enforce either behaviour.
The part I'd want defined before calling that a contract is the equivalence relation for the router test. Recomputing a recorded choice and comparing it with the recorded one assumes "different" is meaningful — but a tie between two routes, a reordered candidate list, or a threshold read from a value that has since been revised can all produce a legitimate difference. Without a rule for which differences count as a regression, replay degrades into a string comparison against one historical run, and it will fail on exactly the changes you want it to tolerate. I'd put that rule in the white-box evaluation set: positive cases that must reproduce, boundary cases at the switch threshold, and follow-up cases where an upstream change is allowed to alter the choice.
There's also a closure between your "prohibit live side effects" and the read-surface point from earlier. If the downstream test must not consult the live router, and a rule must not read a live clock or config, then the envelope's boundary is precisely the node's declared read surface. Capture isn't a list of convenient fields — it's the complement of that surface. Otherwise you get a contract that passes locally and fails in production on a read nobody declared.
When a recomputed route differs from the recorded one, how do you classify it: regression, environmental drift, or allowed non-determinism — and does that classification live in the trace or in the reviewer's head?
The distinction between where a decision is written and where it runs is where most agent systems fall down.
When teams put operational constraints into markdown prompts, they treat the LLM as both the actor and the policy enforcer. If the context window fills up or the prompt drifts, the model negotiates with its own instructions. The failure looks like a hallucination, but the architectural flaw was asking next-token prediction to act as an access control list.
In my own harnesses, the cleanest boundary has been treating the model strictly as an untrusted generator of proposed edits or tool payloads. Every invariant that matters (filesystem boundaries, API allowances, schema validation, and state progression) lives in the shell wrapper or middleware. If a tool call exceeds scope or misses required fields, the runtime halts it with an exit code before any state mutates. Having a deterministic receipt on disk means a failed run can be replayed and tested locally without paying for twenty conversational turns to hit the same bug.
The actor / policy-enforcer split is the cleaner way to state the boundary: it makes the question about where a rule is enforced, not how strongly it is worded. That is exactly what a deterministic backbone is for — the model proposes, code decides, and the receipt records both.
One thing the wrapper design tends to leave out: if it only validates the payload's shape, a replay can still diverge. The same tool call can be legal under one state revision and illegal under the next, and schema validation won't see it. So the recorded input has to include the state version the rule read, not just the arguments it was passed.
The other gap is on the rejection path. If the runtime halts an out-of-scope call before any state mutates, the attempt often never reaches the log — which means the audit trail shows a clean run and can't tell you whether the rules held or the model simply never tested them. Recording rejected payloads alongside accepted ones is what turns a receipt into evidence; otherwise you're replaying only the branch that happened to work.
Do you record the payloads your wrapper rejects, or only the calls that went through?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.