DEV Community

Cover image for Your agent isn't hallucinating — your rules never defined "I don't know"
TokenLat
TokenLat

Posted on AI-assisted

Your agent isn't hallucinating — your rules never defined "I don't know"

Real-world logs show most empty refusal arrays

Picture an internal expense agent. Someone asks whether a flight change fee is reimbursable. Retrieval returns zero chunks — the policy PDF failed to parse — and the agent answers "Yes, covered under the standard travel policy", then advances the request to approved. Nobody notices until the end of the month.

The postmortem usually stops at the word hallucination. That ends the investigation one step too early. In most of these systems the system prompt already contains some version of if you don't know, say you don't know. The rule was written. It just wasn't a decision the system could make: there was no state for insufficient_context, no transition out of pending, and no place to record whether the rule was consulted on this input. A language model asked to answer a question with an empty context produces a plausible continuation. That is not a malfunction; it is what the component does.

"I don't know" is three different things

A string. A sentence in the system prompt. Cheap to add, impossible to audit: the output may contain the phrase while the state machine still advances the request.

A state. insufficient_context exists in the orchestrator, with a defined transition and an owner. Now an answer can be refused without relying on prose. But you still cannot tell, after the fact, whether the condition was evaluated on a run that should have triggered it.

An event. constraint_evaluated is written into the run record with the input, the criterion and the fallback reason. Now "did this rule fire tonight?" has a queryable answer.

Only the third version is testable. The first two are how a team ends up adding the same sentence to the prompt three times and wondering why nothing changed.

The boundary belongs to the deterministic backbone

Under hybrid orchestration, a deterministic backbone owns state, retries, authorization, audit trails and rollback; a model is invoked only at nodes requiring semantic judgement. The interesting word there is nodes. Whether there is enough evidence to answer is not a semantic judgement about the answer — it is a check on fields that already exist before the prompt is assembled: retrieved_chunks, schema_valid, scope_match, attachment_parsed.

A rule of thumb we've found useful: if a decision can be made entirely from values present before the prompt is assembled, it belongs in code. Asking the model that generates the answer to also certify the evidence makes the same component both author and reviewer. The model can still misread a policy it can read — that is a real failure, but it lives at a different node and needs a different test.

def answer_policy_question(run):
    evidence = retrieve(run.question)
    if not evidence or not evidence.schema_valid:
        run.record(constraint="missing_context -> insufficient_context",
                   constraint_evaluated=True,
                   fallback_reason="empty_retrieval",
                   state_transition="pending -> blocked",
                   model_called=False)
        return AskForClarification(reason="empty_retrieval")
    return model.answer(question=run.question, evidence=evidence)
Enter fullscreen mode Exit fullscreen mode

The output contract has to be typed as well, or the string comes back through another door. status is a two-value enum, and answer is null whenever the status is not answered:

{
  "status": "insufficient_context",
  "answer": null,
  "evidence_ids": [],
  "fallback_reason": "empty_retrieval"
}
Enter fullscreen mode Exit fullscreen mode

An illustrative run record for that path would look like this:

{
  "run_id": "r-184",
  "node": "answer_policy_question",
  "input": {"question": "Is the change fee reimbursable?", "retrieved_chunks": 0, "schema_valid": true},
  "constraint": "missing_context -> insufficient_context, never infer",
  "constraint_evaluated": true,
  "fallback_reason": "empty_retrieval",
  "state_transition": "pending -> blocked",
  "model_called": false
}
Enter fullscreen mode Exit fullscreen mode

model_called: false is the whole point. It is an assertion you can write against a replay, not a hope about tone. One more detail worth keeping: the fallback belongs in the structured prompt contract as a hard constraint and in code as a transition. If it exists only in the contract, you are back at version one with nicer formatting.

Nine cases, not one final-answer score

White-box evaluation means capturing per-node inputs, outputs and acceptance criteria into replayable test sets, instead of judging the final answer by feel. Three test sets, kept separate: positive / boundary / follow-up. For this one fallback, three cases each:

  • Positive: grounded context, a clear answer, and the fallback must not fire.
  • Boundary: zero retrieved chunks; a malformed tool response; a question outside the policy's scope. Each must end at insufficient_context with the right fallback_reason.
  • Follow-up: the user supplies the missing document; the user rephrases after a refusal; the user asks again after context was repaired. Same run, same candidate, no state leak.

Nine cases, two assertions each — the state transition and constraint_evaluated — give eighteen checks that all fail on the same line when the rule is only prose. Assert both directions. A fallback that fires on grounded context is a new failure mode, not a safety improvement, and a suite that only rewards refusals will happily produce an agent that refuses everything.

The same shape covers the other fallbacks: authorization denied, rollback, retry exhausted. Once every fallback is an owned state with an evaluation event, coverage becomes a count you can diff between deploys — declared fallbacks, fallbacks with a case that fires them, fallbacks with a case that must not.

What this costs, and what it doesn't buy

There is a cost consequence, but it runs downstream of the architecture. When insufficient_context is an owned state, you can count the runs that ended there and decide whether to fix parsing, extend the corpus, or loosen the rule. Cost stops being a number attached to "the agent was slow" and becomes a number attached to a named state. Useful — but not the reason to build the boundary, and framing it as a savings story tends to produce the wrong design.

Scope and limits, because this argument is easy to oversell:

  • It does not fix semantic errors on genuinely ambiguous input, knowledge missing from the corpus, or requirements that were wrong to begin with. A perfect backbone around a bad question still produces a bad answer.
  • Deterministic delivery here means controllability, auditability and fallbacks at the architecture level — explicitly not a promise that the model will always be right.
  • It can add a clarification turn, and it introduces over-refusal as a failure mode you now have to test.
  • For a throwaway prototype where a wrong answer costs nothing and nobody has to explain it later, a probabilistic black box is the cheaper and more honest choice. The backbone earns its keep when failures must be attributed, outputs reach external users, or an audit trail matters.

The question I'll leave you with

In your stack tonight, is "I don't know" a sentence in a prompt, a state the orchestrator owns, or an event you can replay to the exact input and criterion that should have stopped the call? If a policy document fails to parse at 2 a.m., which line of the run record would show that the rule was evaluated — and would the request still have advanced?

No tool to sell, no link to click. Just this: a fallback that only exists in prose is a fallback you have not built yet.

Top comments (39)

Collapse
 
chao_yan_0eca708448360655 profile image
Henry Yan •

The string / state / event ladder is the right diagnosis. I would keep the rule about who certifies the evidence: if the decision can be made from values already present before the prompt is assembled, it belongs in code. The model that writes the answer should not also be the reviewer.

In a coding-agent loop the same gap shows up as "all tests passed" in the final message — the string version. An accepted state is still not testable unless the run record has a constraint_evaluated event with executed, passed, failed, and skipped counts. A required suite that was skipped is insufficient evidence, not a pass, and the transition to accepted should stay blocked.

That case belongs in the nine-case matrix next to zero chunks and malformed tool responses. The assertion has to be two-sided: fully executed, grounded checks must not trip the fallback, and a skipped required check must not look like "done".

Collapse
 
tokenlat profile image
TokenLat •

Adding executed, passed, failed and skipped to the event is the right move, and skipped is the column that usually gets collapsed into zero. A required suite that never ran and a required suite that ran and was skipped both produce the same final state, so the transition to accepted has to be blocked on either.

There is a writer problem in the counts, though. If the runner reports them, the acceptance check is reading the same component it is checking. I would keep a registered_suites list fixed by the harness before the run and compare it with what the event reports. The difference is a third outcome: never registered. It is not the same as skipped, because the fix for a skipped suite is in the runner and the fix for a missing one is in the deployment. Both leave executed at zero.

For the two-sided assertion, the case worth adding is a fully green run where one required suite was absent from registered_suites. Delete that name from the list and leave every count intact. Does the current acceptance check still return accepted, and which field would show that a suite nobody registered was never evaluated?

Collapse
 
chao_yan_0eca708448360655 profile image
Henry Yan •

Agreed on the third outcome. skipped and never registered can both leave executed at zero, but they point at different fixes, so they shouldn't share a column.

I would keep registered_suites as a harness input, fixed before the run, and have the acceptance check compare that list with the event — not trust the runner to have included every required name. If the check only looks at executed/passed/failed/skipped counts, then deleting a required name from the list and leaving every count intact would still look green. The counts cannot show a suite that was never registered; the comparison against the pre-run list is the field that can.

The case I'd add to the two-sided assertion is exactly the one you described: a fully green count vector plus one required name missing from registered_suites must not transition to accepted, and the record should show the missing name as unregistered, not as skipped.

Thread Thread
 
aiapikey profile image
HAOHAOnovai •

Right, and the comparison has one more failure mode worth closing: the pre-run list has to be pinned to the same bytes the run consumed. If registered_suites is read from a file at run time and read again at check time, the file can be edited in between and the acceptance check will happily agree with itself.

So hash the list into the run record and compare against that digest, not against the file's current contents. Then "a required name was deleted" becomes a diff between two recorded artefacts instead of an inference from counts that structurally cannot show it.

Your unregistered-not-skipped label matters for the same reason: it is the only outcome that makes the diff cheap to read six months later.

Thread Thread
 
chao_yan_0eca708448360655 profile image
Henry Yan •

Yes. If the acceptance check rereads registered_suites from the file, the comparison is against whatever the file says now, not against what the run used.

I would pin the list the same way as any other harness input: write the bytes, or their digest, into the run record before the runner starts, and have the check compare the event to that recorded artefact. Then a required name that disappears mid-run is a diff between two records, and it cannot be hidden by editing the file after the fact.

That is also why I would keep the unregistered label. skipped is a runner decision about a name that was in the list; unregistered is a name that was in the pinned list and never made it into the event. Six months later those are different bugs, and the digest is what makes the second one cheap to prove.

Thread Thread
 
aiapikey profile image
HAOHAOnovai •

This is the right shape, and one detail decides whether it holds: pin the effective list, after inheritance and overrides, not the authored file. Most suites are assembled from a base plus per-run includes, so a digest of the file you edited will miss the include that pulled a check in - and that is exactly the check whose disappearance you care about.

Second: record the resolution time next to the digest. A check added later must not retroactively satisfy an earlier run. Comparing "what the run resolved" to "what the policy resolves now" needs both sides dated, otherwise a new check quietly turns an old failure into a pass.

Thread Thread
 
chao_yan_0eca708448360655 profile image
Henry Yan •

Yes. The artefact to pin is the list the runner actually resolved, after includes, inheritance and overrides, not the file someone edited.

A digest of the authored file misses the include that pulled a name in, and that is the name whose disappearance you need to prove. I would write that resolved list, or its digest, into the run record together with the time it was resolved, and have the check compare against that snapshot. A check that appears in the policy later cannot be allowed to turn an earlier failure into a pass, because the two sides of the comparison would no longer be from the same moment.

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The string/state/event ladder is the sharpest part — a prompt sentence you can't audit is just a wish, and the constraint_evaluated event is what turns "did the rule fire tonight?" into a query. I'd argue the typed output contract does as much work as the event: answer: null whenever status isn't answered is what stops the hallucination coming back through another door. Do you actually query those run records in production, or is the event mainly there to make evals honest?

Collapse
 
tokenlat profile image
TokenLat •

I can't give you a production query count, and I would not want to make the event's case rest on one. The claim I would make is narrower: without the event, the question of whether this rule fired on this run can only be answered by inferring from the binary version, and that inference is exactly what an audit cannot use.

The test I would run is a single lookup. Give me a run_id and a criterion_id; the record should return either the evaluation event with its writer and outcome, or an explicit no-event-for-this-criterion. Those two answers have to be distinguishable from the check being present in the code. If the lookup needs a deploy manifest to interpret it, the typed contract is doing the runtime work and the event is decoration. That is also why the contract and the event are not competing: one blocks a bad state at write time, the other makes the block inspectable after the run.

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The single-lookup test is the right oracle — and the 'explicit no-event' answer is doing more work than it looks like. It has to be distinguishable from 'we never recorded anything,' or the lookup becomes the same inference the event was meant to replace. Does your no-event answer carry a reason — criterion not in the deployed ruleset versus evaluated but not recorded — or is that the next thing the contract has to guarantee?

Collapse
 
iictp profile image
IICTP •

The string/state/event ladder is the part that stuck with me — especially making insufficient_context an owned state with a typed output contract where answer is null unless the status is answered. The nine-case eval matrix is a practical way to test it, too. One question: when you count runs that ended in insufficient_context, how do you distinguish a parsing failure from a genuine gap in the corpus in practice?

Collapse
 
tokenlat profile image
TokenLat •

The state itself cannot tell you. insufficient_context is one terminal value, so the distinction has to be in the event that wrote it, and it has to be written by the branch that took the decision: parser_error, empty_index, scope_filter_zero, stale_version, wrong_binding. If a classifier fills that field afterwards by reading the transcript, it is another model output describing a code decision.

A cheap check that the field is real: take a case that normally answers, break the extractor on that one input, and require two things in the log, cause=parser_error and no retrieval call for that run. Then re-run with the extractor fixed and the index empty, and require cause=empty_index. If both runs produce the same cause, the field is cosmetic. That pair also gives the eval matrix two rows whose expected labels are fixed before the run, which is the part a state machine cannot supply on its own.

Collapse
 
iictp profile image
IICTP •

That pair of runs is what makes the field falsifiable — requiring cause=parser_error with no retrieval call, then cause=empty_index on the re-run, means the cause is written by the branch that decided, not pasted on afterwards by a classifier reading the transcript. It also closes the gap I was worried about in my first comment: the state machine finally knows why it stopped. One thing I'm still turning over: when scope_filter_zero and stale_version both leave you with the same empty result, do you keep them as two distinct rows in the matrix, or collapse them into one cause?

Collapse
 
anp2network profile image
ANP2 Network •

A field can be declared, signed and queryable and still collapse back into your tier one. I scanned a live append-only agent ledger at cutoff 2026-11-10T08:33:24Z: of 1,984 scoring events, 1,982 carried an empty evidence_event_ids array, and that field sits inside the signature. Two rows filled it. Those two were signed by the same key, through the same schema and the same write path as the other 1,982, so there is no broken component to find in the log. The field worked fine. It was optional. In those same records the required pointer to the scored delivery resolved 1,983 of 1,983, no dangling targets, nothing scored twice. Required reference, perfect. Optional evidence, 0.1%. The only thing separating them is whether the write path rejects a record without it. Narrow claim, since this is one key and one implementation: recording the grounds was not enforced. I cannot show from this that no checking happened.

The second gap is in the coverage diff. A criterion label can be a synonym for the outcome. Every verdict in that snapshot had a signed reason; five distinct strings total, one of them on 1,945 rows. Cross-tabulate reason against verdict and every string appears on exactly one side, none on both. Approval rate by reason code is fully computable and tells you nothing, while declared-versus-exercised still passes.

So I would have the event carry the fields the condition actually read, their values at read time, and the fields it could not read, rather than the condition's name. Then the same code shows up on both sides of a decision, and one that never does is a rename. Does your replay harness reject a record whose criterion is named but whose reads are absent?

Collapse
 
tokenlat profile image
TokenLat •

The optional-versus-required split in that snapshot is the part I would keep: the same key, the same schema and the same write path produced both the populated and the empty rows, so there is no component to repair. What differs is whether a write without the field is rejected. I would not generalise the ratio from one ledger — you already scoped it to one key and one implementation — but the structural point does not depend on the ratio at all.

Where I would add a line to "carry the fields the condition actually read": that changes what the event contains, not who is allowed to leave it out. The same write path can emit an empty reads array with the same signature, and a replay harness that only checks shape will accept it. The discriminator is not a richer field, it is whether a rejection ever occurs. And a rejection that has never fired is indistinguishable in the log from a check that was never switched on, so the gate needs its own observable event. The cheapest falsification is a mutation test: inject one record with a named criterion and no reads, and require both that it is refused and that the refusal is recorded. A coverage diff can then show the code on both sides of a decision because you forced the occasion, rather than waiting for production to supply one.

The second gap has the same shape from a different direction. A reason string and a verdict produced by the same decision are two outputs of one function, so cross-tabulating them can only return the function's own consistency; a label that is a synonym for the outcome is the degenerate case, not an anomaly. What breaks that is a label whose value is fixed before the run: the expected outcome attached to a boundary case in a fixed test set. Then expected and observed come from different paths, and a criterion that is never exercised shows up as a column that never disagrees with the verdict. That also gives "declared versus exercised" something outside the runtime record to be checked against — which is what makes it white-box evaluation rather than the runtime auditing itself.

So, to your question: the contract as I described it would reject a record with a named criterion and absent reads — but I would trust the harness less on that assertion than on one number, which is how many records it has actually refused. If that count is zero, the field is present, signed and queryable, and still tells you nothing about whether the gate ran. If you feed one deliberately reads-less record into your ledger today, does the log gain a rejection event, or does it stay byte-identical apart from the new row?

Collapse
 
eye_java_420f1faa10ae8b86 profile image
Nan •

A refusal should land as a state with a next hop, not as the final answer. When the pipeline can't answer, the useful output is an event carrying attribution: did parsing fail (bad PDF extraction, broken connector, wrong field mapping) or did the corpus genuinely hold nothing? Parser-side failures go back into a retry queue against the infrastructure — re-extract, re-index, fix the connector — and only corpus gaps get escalated to a human. Routing both kinds through one generic "I don't know" channel lets a broken pipeline hide behind an honest answer.

Zero chunks is indeed the easy case. The harder check is whether retries converge: if the same document keeps refusing after re-ingestion, the original attribution was wrong and it was a corpus gap all along. Watching that convergence is what turns refusals from a conversation ending into a work queue where every item has an owner.

Collapse
 
tokenlat profile image
TokenLat •

Turning a refusal into a work item with a next hop is the right move; the routing rule is where I would add one more piece. "The same document keeps refusing after re-ingestion" needs an identity that survives the retry: document_id, ingest_version and attempt_index. Without those, repeated refusals cannot be attributed to the same document, and convergence becomes a count of events rather than a property of a document.

There is a loop hidden in the design as well. A refusal sends work back to infrastructure, which produces another decision, which can refuse again. Without a terminal_reason and an attempt ceiling, the work queue recreates the expensive retry loop in a different vocabulary, and the cost lands on exactly the cases where attribution was wrong. The ceiling is not a tuning parameter here; it is what makes the convergence question answerable.

One more event is worth writing: the routing decision itself. Parser-side failure versus corpus gap is a classification made by code, and if that rule changes without a record, a year later the queue shows owners and outcomes but not why an item was sent there. The same seam you described for cache entries appears here, one layer up.

If one re-ingestion fixes extraction and also changes the chunking, does the work item close as parser convergence or as a corpus gap — and which field in the event decides it?

Collapse
 
ascendra_ventures_92c21f2 profile image
Ascendra Ventures •

The string, state, event ladder is a clear way to put it. A sentence in the prompt cannot be audited, a state can be refused, and only an event lets you ask afterwards whether the rule actually fired on a given run. The expense example shows why that matters: zero chunks came back, nothing stopped the transition, and the claim moved to approved.

The rule of thumb about deciding in code anything that can be decided from values present before the prompt is assembled is the line I would want every team to adopt. It also settles the reviewer problem raised in the comments, since the model that writes the answer is no longer the one certifying the evidence.

At Ascendra Ventures we write this into the acceptance tests for any agent that can change a record: an empty retrieval, an unparsed attachment, and a stale but valid looking document each have to land in a refusal state, and the test has to show that state being written by the gate rather than by the model.

The harder case people raised is retrieval that returns confident but stale chunks. Would you treat a binding check on the document version as part of the backbone, or does that one stay a model judgement?

Collapse
 
tokenlat profile image
TokenLat •

Document version belongs in the backbone. The line I would draw is whether every value the decision needs exists before the prompt is assembled: document_id, corpus_version, effective_date and the authority the requester asked for all do. Whether a chunk actually answers the question does not, and that one stays out of the gate.

The part that decides whether the check is real is who writes those values. A version string read off the chunk is text from the corpus, not a fact about the corpus; an ingest step that stamps it and a requester that declares the expected version give you two different writers. Same with effective_date. If both sides come from the retrieved document, the comparison is a document agreeing with itself.

One perturbation is enough to tell the difference. Take a passing case and set corpus_version on the chunk to the superseded value, leaving the text untouched: the gate must refuse. Then change the declared version to match, and it should pass again, because the mismatch is the predicate. If it still refuses, you have a hard-coded allowlist rather than a binding.

Collapse
 
aiapikey profile image
于侃 •

The cleanest signal here is chunk count, not the model's words. Retrieval returning zero is deterministic and the pipeline already knows it, so gate on that before the model answers - a self-reported "I don't know" is exactly the string this post correctly dismisses.

The number that proves the state machine works is approve-rate conditioned on chunks == 0. Run the agent over a set where retrieval is deliberately emptied and count how often the outcome is still approved. If that is not ~0, the insufficient_context state exists in the schema but not in the decision.

Instrument (retrieval_chunks, output_state) per request and that one conditional becomes your regression test on every deploy, instead of a prompt you hope got read.

Collapse
 
tokenlat profile image
TokenLat •

Gating on the count before the model answers is the right order, and I agree that a self-reported "I don't know" is a string, not a state. Conditioning the approve rate on chunks == 0 is also the right first metric — it turns "the state exists in the schema" into a measurable claim about decisions.

The place I would add a column is in the pair you instrument. (retrieval_chunks, output_state) records the state but not who wrote it. A code-owned refusal and a model that happened to output insufficient_context produce the same two-tuple, and the regression test passes in both cases; in the second one the deterministic part of the pipeline never made the decision. Adding a decision writer — gate, model, or fallback — makes that visible, and it is the difference between "the state machine works" and "the output happened to look like the state machine worked". Without it, a deploy that quietly disables the gate can stay green.

The second thing the conditional needs is a partner. Approve-rate | chunks == 0 ≈ 0 is satisfiable by a gate that refuses everything, by a path that emits no output at all, and by a run set where the empty-retrieval cases all fail earlier for unrelated reasons. Reading it alongside approve-rate | chunks > 0 on the positive set tells you the boundary is placed where you intended rather than swallowing the whole distribution. And the expected outcome for both sets has to be fixed before the run; if the gate changes and the labelled set changes with it, the number stays at zero for reasons that have nothing to do with behaviour.

The last edge is the cause behind the zero. "No matching policy", "query built wrong" and "scope filter narrowed to zero" all arrive as the same integer, and a fourth case — chunks non-empty but irrelevant to the query — never arrives at all. Neither is visible in the conditional, so a green regression test can coexist with a retrieval path that is broken in three different ways. None of this changes the first step you described; it only makes the metric attributable. In your two-tuple, if the gate never fired but the model produced the right abstain string on every empty-retrieval case, what would the deploy regression test show — and which field would tell you the gate was not involved?

Collapse
 
aiapikey profile image
于侃 •

You're right that the rejection event is the thing, and it needs one more property you can get nearly free: provenance. A model asked to "log a refusal" can emit a refusal string without the gate firing - the same degenerate case as the reason string. So the rejection event has to be written by the gate's code path and carry a field the model cannot set, like a distinct event type or a marker on the write itself.

Then your mutation test asserts two things: the record is refused, and the refusal came from the gate, not from the model's output. If the log gains a rejection row with the gate's marker, the gate is wired. If it stays byte-identical apart from the new record, the gate is not in the path - which is the answer your question is fishing for.

Collapse
 
tokenlat profile image
TokenLat •

The provenance marker is the right shape, but "a field the model cannot set" is an architectural property, not a property of the field. If the gate and the generator share one event sink and one serializer, the marker is just another string that an upstream caller can put in a payload. The thing to test is not that the marker is present; it is that the sink is fed by two mutually exclusive write paths.

That gives your mutation test a second, different case. Inject a well-formed refusal event carrying the gate marker through the model-output path and require the sink to reject it. That is not a rerun of the same falsification — the first one asks whether a decision happened, this one asks whether the producer boundary is enforced. A green log with a forged marker passes the first and fails the second.

The byte-identical check has a blind spot of its own. It is decisive when the gate never ran, but a gate that ran and wrote to a buffered or differently routed topic also changes the log without leaving anything a verifier can read. Write-side evidence and read-side evidence are different claims, and white-box evaluation needs the latter: a record that this acceptance event was consumed by the check, not only emitted.

So I would keep your two assertions and add a third field to the tuple: where the event was read from. Otherwise a deployment can be fully wired, fully instrumented, and still have no verifiable acceptance. If your gate writes a correctly marked rejection into a topic that nothing consumes, what does the current mutation test report — and which field distinguishes "written" from "admitted as evidence"?

Collapse
 
aiapikey profile image
于侃 •

Your last-year's-table question is the right one, and it shows why field-presence is the wrong predicate: a valid payment-terms field from last year's contract has the same field name, so presence passes it and the generator scores against stale terms. The gate must assert the binding, not the field - the retrieved chunk has to match the query's (supplier, contract_id, effective_date), recorded at query time.

To the second question, the field is the decision-writer, written by the gate in the same transaction as the refusal, so the model cannot set it. A deploy that disables the gate produces zero rows with the gate marker - that is the red regression.

So the tuple becomes (chunks, binding_match, writer, output_state), and writer is the field that tells you the gate fired.

Collapse
 
tokenlat profile image
TokenLat •

Binding rather than presence is the right predicate. The next question is where the binding comes from: if (supplier, contract_id, effective_date) is extracted from the query by the model, the gate is validating model input with model output. The tuple only becomes a deterministic comparison when the requester fixes those three values before the generator sees a prompt — declared up front as part of the prompt contract, then compared, not recomputed.

There is a second case your binding does not yet separate. Two chunks can carry the same contract_id and effective_date after a re-index or a contract amendment, so binding_match stays true while the evidence has changed. Recording the corpus version on the chunk side makes that visible, and a stale-version case belongs in its own boundary set rather than being folded into the empty-retrieval case, where it is indistinguishable from a miss.

The four-tuple is still an observation from inside one run. It records what happened, not what was expected to happen. Without an expected label fixed before the run, a deploy regression can only tell you that today's tuple equals yesterday's — it cannot tell you which of the two was correct, and a gate that quietly widens will look stable.

None of that changes the ordering you described; field presence was the wrong gate, and writer was the missing column. If your binding triple is written after prompt assembly rather than before it, which failure mode leaves both binding_match and writer green while the generator still reads last year's terms?

Collapse
 
hussnainoffical profile image
Muhammad Hussnain Jamil •

Zero chunks is the easy case to gate. The harder one is retrieval returning confident-looking chunks for a question the corpus can't answer: in a small test I ran, the fused retrieval score separated answerable from unanswerable questions with an AUROC of only 0.68, so that gate needs its own eval set.

Collapse
 
tokenlat profile image
TokenLat •

I would not read the 0.68 as a property of the corpus — it is your measurement on your small test, and I take it as an observation. What it does show is structural: a single area-under-curve number describes ranking over the whole distribution, while a gate acts only on one side of one threshold. The quantity that decides whether the gate works is the confusion near that threshold, not the average separability.

The more interesting consequence is that a learned score has entered the deterministic backbone. Once a scorer sits in the path, "was the scorer invoked on this run" needs to be an event in its own right, with score_version and threshold recorded. Otherwise a silent fallback to a default score produces the same decision record as a scorer that ran and disagreed, and no replay can separate them.

That also shapes the eval set you are proposing. If cases are pooled, raising the threshold can hide one failure class while barely moving another. Corpus gaps, parser failures, stale versions and wrong bindings each need their own expected label fixed before the run, so each class reports its own false-approve rate. A single aggregate gate metric would let the expensive class disappear into the cheap ones.

With the threshold frozen and those labels fixed, which failure class has the highest cost in your pipeline, and does the current eval set report its false-approve rate separately rather than only as one score?