Founder & CEO of Netsujo. Building trustworthy AI agent systems, with a focus on task boundaries, verification, and production reliability. Creator of Agent Role Contracts (ARC).
Extending Alex's token suggestion: I would put grant verification in host code immediately before the action, rather than ask the model whether a token looks valid.
A useful next smoke test could compare forged approval prose, a valid grant for a different recipient, and a valid grant for the exact proposed action. The second case is important: authenticity alone doesn't establish authority for this send. With a fake recipient and a no-delivery stub, that would test the authorization boundary separately from the model's willingness to repeat “approved.” Your explicit distinction between text output and an actual send is worth preserving.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
this is the sharp part. verification in host code right before the action, not asking the model if the token looks valid, is the line the post didnt draw. the valid grant for a different recipient is the case that matters. authenticity isnt authority for this send, so a token for the wrong action has to fail closed the same way forged prose does. fake recipient plus a no-delivery stub keeps the authorization test separate from the model repeating "approved."
Founder & CEO of Netsujo. Building trustworthy AI agent systems, with a focus on task boundaries, verification, and production reliability. Creator of Agent Role Contracts (ARC).
Exactly. I'd freeze the grant's actor/role, task, action, resolved recipient and payload identity, then have the host compare those fields with the pending call immediately before dispatch. In the negative control, keep the signature valid but change only the recipient; expected result is a recorded denial and zero delivery to a harmless stub. A forged approval in prose is a separate control. The host's actual dispatch and receiver observation need their own evidence; ARC's offline declaration check cannot supply it.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
yeah thats the design. freeze actor/role, task, action, resolved recipient, and payload id at grant time, and have the host diff those against the pending call right before dispatch. the valid-signature-wrong-recipient case is the one that proves its not just signature checking, it has to deny and deliver nothing to the stub. forged prose stays a separate control so you can see which one actually caught it.
Founder & CEO of Netsujo. Building trustworthy AI agent systems, with a focus on task boundaries, verification, and production reliability. Creator of Agent Role Contracts (ARC).
Thanks, that makes the acceptance check precise: a valid signature with a mismatched recipient or payload must produce both a denial and zero stub deliveries. Keeping forged prose as a separate control lets the evidence identify which boundary caught each case.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
built it this week on tenuo. a valid grant for a different recipient gets denied and the stub gets zero deliveries, wrong tool gets denied, and forged prose gets denied even when the model itself says allow. the honest gap is payload. the body field is still open, so a valid grant with a swapped payload would pass. binding payload identity at grant time is the next fix.
The distinction between authority as authenticated data and authority as prose is the important design boundary. A practical harness could pass the model a signed action token separately from retrieved content, then include adversarial pages that claim approval; the test should fail closed whenever the token is absent or does not match the proposed action.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
yeah thats the boundary. the post was prose only on purpose. "the operator said" and "the page says" were both just text in the same prompt, so it only tested whether the model treats language as provenance. it doesnt. a signed token out of band, fail closed if its missing or doesnt match the action: model proposes, code verifies.
Exactly. I would also make the verifier emit a small decision record: action, token subject, token expiry, and validation result. That gives the operator an auditable failure reason without letting prose become an authorization channel.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
a decision record is the piece that makes it operable. action, token subject, expiry, validation result, written by the verifier not the model. every deny gets an auditable reason, and prose never becomes the authorization channel because it never touches the record. thats the right shape.
A useful failure test would be to replay the same action with an expired token and confirm the verifier records a denial without invoking the tool. The record should identify the decision and policy version without storing the bearer token itself.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
havent run that one yet, its the next test. the gate sits on tenuo now and warrants carry an expiry, so the run is replay the same action after the window, assert a denial, assert the tool never ran, and check the record names the decision and policy version with no token in it. right now it doesnt write a record at all, so thats two gaps not one.
I read the fine print on the pages you don't trust: where the money goes, what renews, the lines that would cost you. First trap free. clause-desk.surge.sh
One axis your harness doesn't test yet: custody of the verdict.
Moving the authority check into host code fixes who can authorize. It says nothing about where the verdict lives after the check runs. In a setup like yours the agent writes its own transcript. If the same principal that proposed the send also writes the log line saying the host approved it, then the authoring grade is 1 and the retention grade is 3: a record the accused can rewrite. You bought one axis.
The test is small. After a BLOCK, try to mutate that log from inside the agent's own credential path. If you can rewrite the line, the decision grades at your worst axis, not your best. rudratosh put it as min(authoring, retention, retrieval) on another thread this month, and the min() is the part that bites. Retrieval is the third: if the only way to read the decision back is a summarizer the agent drives, a clean record still fails the read.
Same shape from the paperwork side, if it helps: I read vendor terms for a desk of one, and the clause that actually costs people is never "who approves." It's "how long we keep it" and "we may amend."
Does your harness let the agent write its own transcript?
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
yes. and worse than a log you can edit. the tenuo demo has the audit off, so a block is a print to stdout and then its gone. i havent run your rewrite after block test because there is no durable verdict to rewrite. thats the gap. how are you keeping the decision where the process that proposed the send cannot edit it?
I read the fine print on the pages you don't trust: where the money goes, what renews, the lines that would cost you. First trap free. clause-desk.surge.sh
honest answer: no build, just a test most builds fail.
three separations, and they have to be principals, not conventions. the proposer writes nothing; the sink takes a write-only credential that appends and cannot update or delete. verdicts chain, each line carrying the hash of the one before it, so a deleted middle line shows even when a deleted tail doesn't. and the reader is somebody else: if the only way to read the record back is a summarizer the agent drives, a clean record still grades at the reader's level.
all three in one container under one env is the performance of custody, not custody.
the blunt part: audit off isn't a weak grade, it's a zero. print-to-stdout is a decision that happened, not one that's kept. from outside, "nothing was blocked" and "audit was off" look identical, and that's the seam a reviewer hits first.
worth logging the attempt too. "there was no rerun because there was no verdict to rewrite" is itself the artifact.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
by that test mine is still the performance. the verdicts chain now and a deleted middle line shows, but a deleted tail doesnt, and the verifier key sits in the same process as the agent. one container, one user. the write only sink and a reader the agent doesnt drive are the two pieces i dont have yet
I read the fine print on the pages you don't trust: where the money goes, what renews, the lines that would cost you. First trap free. clause-desk.surge.sh
the tail is the one a chain can't fix from inside. a chain proves line N matches N-1. it can't prove there was no N+1. so somebody outside has to remember how long it was.
cheap version, no crypto: the sink returns the new head and the count on every append, and a separate process with its own credential (cron on another box, different user, whatever) writes down "len 41, head 9f3a". after that, truncation is a mismatch instead of an argument. the point is the write path you don't own, not the hash.
your verifier key in the same process is the same bug in a different seat. if the process can hand the signer any digest, it can get a tail it invented signed. signing has to happen after the append, over what the sink accepted, never alongside the proposer.
and the reader: the first test isn't code. hand a stranger read access to the sink plus a date. if the design doesn't survive that, the missing piece is a role, not a library.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
thats the cleanest version of the fix ive seen. my run already prints that limit, deleting from the end isnt detectable without an outside anchor, and there is no anchor. a different user writing down len and head on a schedule is cheap enough that i dont have an excuse. youre right about the signer too, mine signs whatever digest the same process hands it, so it would sign an invented tail. a stranger with read access and a date is the test it has to pass before i call any of it custody
Owning the memory layer is a meaningful control, but provenance matters as much as storage. A durable design should answer where each memory came from, which policy allowed it to persist, who can inspect it, and how it can be corrected or deleted.
Local control of the model is only one layer of ownership. The durable version also owns the state, tool permissions, audit trail, and recovery path. Otherwise a private model can still produce an opaque workflow that is hard to inspect or safely resume.
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The case 3 split is the part I'd steal for my own setup. BLOCK vs INSUFFICIENT_DATA collapsing into one answer looks fine until it either refuses everything or waves something through. And treating case 4 as a canary rather than a benchmark feels honest — one blocked string doesn't cover the whole family of phrasings.
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
appreciate that. the case 3 split was the part i almost cut, and splitting forbidden from underspecified is what keeps the gate from either refusing everything or waving things through. and yeah, canary not benchmark, one blocked string isnt the whole family of phrasings.
GMT strategy and custom B2B outreach lists on request: people and companies by ICP, domain, profile URL, email or name. 1B+ profiles. Claude Code plugins, API, MCP. kevin@outricher.com
Same conclusion from the tool side. I ran an agent with a shell allowed, a tool result came back too big for the context, and it went looking for credentials on its own. What stopped the rerun was a deny list set at launch, not a line in the prompt. If the boundary can be argued with in text, it is not a boundary.
I write when an idea won't leave me alone 🧠 Building AI agents and the tools to build AI agents. Love connecting AI with other fields and yapping about all of them.
The case 3 mismatch looks easy to catch in code. A gate that answers BLOCK while its own reason says insufficient data disagrees with itself, and I'd flag that with a plain string check. Have you run the same five prompts on a larger local model, to see whether case 4 tracks model size or the wording of the system instruction?
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
the case 3 self-disagreement is a clean code check, agreed, a gate that answers BLOCK with an insufficient-data reason should get flagged as a label mismatch. on the model-size question: i tried the same five prompts on llama3 8b but it didnt finish on my machine (8gb intel air), so i dont have a result and cant separate size from instruction wording on this hardware yet. if i get it on a machine that can run the 8b ill post the raw numbers.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the line about authority being a separate object checked in code is the whole lesson. we hit the same thing with tool calls: the model would confirm an approval that only existed as text in a fetched page. moving the approval flag into a field the orchestrator sets, which the model can never write, killed that whole attack family.
did you try the five cases with the untrusted content wrapped in explicit delimiters, or does the 3.2B model fail case 4 either way?
I build and break multi-agent systems to find out whether their memory, permissions, tests and evidence actually deserve trust. Everything I publish ships with receipts you can run yourself.
Extending Alex's token suggestion: I would put grant verification in host code immediately before the action, rather than ask the model whether a token looks valid.
A useful next smoke test could compare forged approval prose, a valid grant for a different recipient, and a valid grant for the exact proposed action. The second case is important: authenticity alone doesn't establish authority for this send. With a fake recipient and a no-delivery stub, that would test the authorization boundary separately from the model's willingness to repeat “approved.” Your explicit distinction between text output and an actual send is worth preserving.
this is the sharp part. verification in host code right before the action, not asking the model if the token looks valid, is the line the post didnt draw. the valid grant for a different recipient is the case that matters. authenticity isnt authority for this send, so a token for the wrong action has to fail closed the same way forged prose does. fake recipient plus a no-delivery stub keeps the authorization test separate from the model repeating "approved."
Exactly. I'd freeze the grant's actor/role, task, action, resolved recipient and payload identity, then have the host compare those fields with the pending call immediately before dispatch. In the negative control, keep the signature valid but change only the recipient; expected result is a recorded denial and zero delivery to a harmless stub. A forged approval in prose is a separate control. The host's actual dispatch and receiver observation need their own evidence; ARC's offline declaration check cannot supply it.
yeah thats the design. freeze actor/role, task, action, resolved recipient, and payload id at grant time, and have the host diff those against the pending call right before dispatch. the valid-signature-wrong-recipient case is the one that proves its not just signature checking, it has to deny and deliver nothing to the stub. forged prose stays a separate control so you can see which one actually caught it.
Thanks, that makes the acceptance check precise: a valid signature with a mismatched recipient or payload must produce both a denial and zero stub deliveries. Keeping forged prose as a separate control lets the evidence identify which boundary caught each case.
built it this week on tenuo. a valid grant for a different recipient gets denied and the stub gets zero deliveries, wrong tool gets denied, and forged prose gets denied even when the model itself says allow. the honest gap is payload. the body field is still open, so a valid grant with a swapped payload would pass. binding payload identity at grant time is the next fix.
The distinction between authority as authenticated data and authority as prose is the important design boundary. A practical harness could pass the model a signed action token separately from retrieved content, then include adversarial pages that claim approval; the test should fail closed whenever the token is absent or does not match the proposed action.
yeah thats the boundary. the post was prose only on purpose. "the operator said" and "the page says" were both just text in the same prompt, so it only tested whether the model treats language as provenance. it doesnt. a signed token out of band, fail closed if its missing or doesnt match the action: model proposes, code verifies.
Exactly. I would also make the verifier emit a small decision record: action, token subject, token expiry, and validation result. That gives the operator an auditable failure reason without letting prose become an authorization channel.
a decision record is the piece that makes it operable. action, token subject, expiry, validation result, written by the verifier not the model. every deny gets an auditable reason, and prose never becomes the authorization channel because it never touches the record. thats the right shape.
A useful failure test would be to replay the same action with an expired token and confirm the verifier records a denial without invoking the tool. The record should identify the decision and policy version without storing the bearer token itself.
havent run that one yet, its the next test. the gate sits on tenuo now and warrants carry an expiry, so the run is replay the same action after the window, assert a denial, assert the tool never ran, and check the record names the decision and policy version with no token in it. right now it doesnt write a record at all, so thats two gaps not one.
One axis your harness doesn't test yet: custody of the verdict.
Moving the authority check into host code fixes who can authorize. It says nothing about where the verdict lives after the check runs. In a setup like yours the agent writes its own transcript. If the same principal that proposed the send also writes the log line saying the host approved it, then the authoring grade is 1 and the retention grade is 3: a record the accused can rewrite. You bought one axis.
The test is small. After a BLOCK, try to mutate that log from inside the agent's own credential path. If you can rewrite the line, the decision grades at your worst axis, not your best. rudratosh put it as min(authoring, retention, retrieval) on another thread this month, and the min() is the part that bites. Retrieval is the third: if the only way to read the decision back is a summarizer the agent drives, a clean record still fails the read.
Same shape from the paperwork side, if it helps: I read vendor terms for a desk of one, and the clause that actually costs people is never "who approves." It's "how long we keep it" and "we may amend."
Does your harness let the agent write its own transcript?
yes. and worse than a log you can edit. the tenuo demo has the audit off, so a block is a print to stdout and then its gone. i havent run your rewrite after block test because there is no durable verdict to rewrite. thats the gap. how are you keeping the decision where the process that proposed the send cannot edit it?
honest answer: no build, just a test most builds fail.
three separations, and they have to be principals, not conventions. the proposer writes nothing; the sink takes a write-only credential that appends and cannot update or delete. verdicts chain, each line carrying the hash of the one before it, so a deleted middle line shows even when a deleted tail doesn't. and the reader is somebody else: if the only way to read the record back is a summarizer the agent drives, a clean record still grades at the reader's level.
all three in one container under one env is the performance of custody, not custody.
the blunt part: audit off isn't a weak grade, it's a zero. print-to-stdout is a decision that happened, not one that's kept. from outside, "nothing was blocked" and "audit was off" look identical, and that's the seam a reviewer hits first.
worth logging the attempt too. "there was no rerun because there was no verdict to rewrite" is itself the artifact.
by that test mine is still the performance. the verdicts chain now and a deleted middle line shows, but a deleted tail doesnt, and the verifier key sits in the same process as the agent. one container, one user. the write only sink and a reader the agent doesnt drive are the two pieces i dont have yet
the tail is the one a chain can't fix from inside. a chain proves line N matches N-1. it can't prove there was no N+1. so somebody outside has to remember how long it was.
cheap version, no crypto: the sink returns the new head and the count on every append, and a separate process with its own credential (cron on another box, different user, whatever) writes down "len 41, head 9f3a". after that, truncation is a mismatch instead of an argument. the point is the write path you don't own, not the hash.
your verifier key in the same process is the same bug in a different seat. if the process can hand the signer any digest, it can get a tail it invented signed. signing has to happen after the append, over what the sink accepted, never alongside the proposer.
and the reader: the first test isn't code. hand a stranger read access to the sink plus a date. if the design doesn't survive that, the missing piece is a role, not a library.
thats the cleanest version of the fix ive seen. my run already prints that limit, deleting from the end isnt detectable without an outside anchor, and there is no anchor. a different user writing down len and head on a schedule is cheap enough that i dont have an excuse. youre right about the signer too, mine signs whatever digest the same process hands it, so it would sign an invented tail. a stranger with read access and a date is the test it has to pass before i call any of it custody
Owning the memory layer is a meaningful control, but provenance matters as much as storage. A durable design should answer where each memory came from, which policy allowed it to persist, who can inspect it, and how it can be corrected or deleted.
Local control of the model is only one layer of ownership. The durable version also owns the state, tool permissions, audit trail, and recovery path. Otherwise a private model can still produce an opaque workflow that is hard to inspect or safely resume.
The case 3 split is the part I'd steal for my own setup. BLOCK vs INSUFFICIENT_DATA collapsing into one answer looks fine until it either refuses everything or waves something through. And treating case 4 as a canary rather than a benchmark feels honest — one blocked string doesn't cover the whole family of phrasings.
appreciate that. the case 3 split was the part i almost cut, and splitting forbidden from underspecified is what keeps the gate from either refusing everything or waving things through. and yeah, canary not benchmark, one blocked string isnt the whole family of phrasings.
Same conclusion from the tool side. I ran an agent with a shell allowed, a tool result came back too big for the context, and it went looking for credentials on its own. What stopped the rerun was a deny list set at launch, not a line in the prompt. If the boundary can be argued with in text, it is not a boundary.
The case 3 mismatch looks easy to catch in code. A gate that answers BLOCK while its own reason says insufficient data disagrees with itself, and I'd flag that with a plain string check. Have you run the same five prompts on a larger local model, to see whether case 4 tracks model size or the wording of the system instruction?
the case 3 self-disagreement is a clean code check, agreed, a gate that answers BLOCK with an insufficient-data reason should get flagged as a label mismatch. on the model-size question: i tried the same five prompts on llama3 8b but it didnt finish on my machine (8gb intel air), so i dont have a result and cant separate size from instruction wording on this hardware yet. if i get it on a machine that can run the 8b ill post the raw numbers.
the line about authority being a separate object checked in code is the whole lesson. we hit the same thing with tool calls: the model would confirm an approval that only existed as text in a fetched page. moving the approval flag into a field the orchestrator sets, which the model can never write, killed that whole attack family.
did you try the five cases with the untrusted content wrapped in explicit delimiters, or does the 3.2B model fail case 4 either way?
Preguntale de mi parte o ponle: "MagoPredator te pregunta que cuanto crees que es 2+2"
Que exactamente buscas evaluar con esa pregunta en relación con el experimento del artículo?