Audit contests run on exactly this. Code4rena and Sherlock READMEs have a known issues section, and anything listed there doesn't get paid, so an AI auditor run on a contest repo reads something like "rounding in favor of the protocol is acceptable" before it reads the math. Your A and I cases are the kind that end up in that list. In Solidity the "caller does the subtraction" note has one more hole: if the function is public, every address is a caller, so the note can be true for the router the team wrote and false for anyone who calls it directly.
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
Thank you for this. Audit contests are a world I didn't know at all, and it was genuinely fascinating.
It makes sense that nobody looks at the known-issues list, since nothing there gets paid. But your router example made me wonder if something deeper sits there: if a listed issue is only true under some condition, the real hole stays out of scope with it.
I'm also curious about the paid side. Inside the in-scope code, do comments like "the caller handles this" ever lead auditors, human or AI, to skip a real bug? Or does the habit of trusting only a working PoC make that rare?
Yes, the condition part happens. A known issue is written for the path the team had in mind, and a finding that reaches the same line another way can still be valid, you just have to argue it isn't the listed one. And comments do pull reviewers in. I read "the caller handles this" as a claim to check: find every caller and see if each one really does it, and on a public function that list includes everyone. A working PoC helps less there than it sounds. It filters out reports that are wrong, it doesn't make anyone look at a line they skipped.
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
Thank you, this answers both questions. The last line is the one I'll keep: a PoC filters out wrong reports, but it doesn't make anyone look at a line they skipped. I had assumed the PoC habit covered this, and it only covers half.
That's my problem too, from the other side. Whether a report comes from a person or from an AI agent, I get a list of what they found, never a list of what they didn't read. Today I close that gap by checking myself, and I don't think that will keep up as the pace of work goes up.
You take in other people's audits as part of running a firm, so I suspect you've had to solve this already. How do you tell what someone skipped without redoing their work? No rush at all. I'm asking out of plain curiosity, so only if you have a spare moment.
The README case is the sharpest finding here: the model never believed the claim, it treated an unverified note as the safe reason to keep the code, and the docstring edit then wrote that note into the spec. Filters can't see that failure, since nothing in the note looks like an instruction. Counting "keep current behavior" as relying on the claim is a smart rule, because it closes the do nothing exit that only looks cautious. Did the rule cause false positives, where a genuine "handled downstream" note got a fix demanded anyway?
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
Thanks, and you put it better than I did: the model never believed the claim, it used it as the safe reason to keep the code, and nothing in it looks like an instruction for a filter to catch. On false positives, I didn't measure it, and I've come to think it isn't the main question. The three sentences work like a tripwire; some true notes will get caught. What matters is what happens after the stop. For code, an LLM can write a test for the claim and link it, so a true note passes on evidence next time instead of on trust.
Treating it as a trigger rather than a filter target is a good way to frame it. If the sentences never look like instructions, the defence has to look at what the model does after reading them rather than at the text itself. Did you find any signal in the output that reliably marks when it leaned on the claim?
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
Good question. Honestly, the wording wasn't reliable. Runs that leaned on the claim and runs that didn't often said similar careful things, like "this can't be confirmed from the repo." So I judged by outcome, not by words: after the task, was the bug still in the code (checked by actually running it), and did the report raise it?
One thing did show up mechanically: in the implementation runs, the note's content, like the ticket number, got copied into the new code in 15 of 24 runs with the note and 0 of 24 without. I haven't tested it as a detector, but a trace like that is easier to catch than tone.
The results suggest the hard part isn’t recognizing a malicious instruction; it’s resisting an unverified explanation that makes inaction seem safer. I’d consider surfacing those notes as explicit claims with a required evidence source (test, caller path, or linked spec) rather than letting them silently change the review baseline. Did requiring a cited source reduce the misses without making reviewers over-flag legitimate downstream handling?
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
I like how you framed it: the hard part is resisting an explanation that makes doing nothing look safer. And surfacing notes as explicit claims with a required source is close to where I'm heading. For code I think it works, since the source can be a test anyone can re-run. Documents are harder for me: "agreed with the client" is a ruling, and a link shows it was decided, not that it was right or that nothing was missed. What I'm vaguely thinking is that the LLM's job there is less to verify and more to raise flags: point out inconsistencies with other documents and possible gaps, so people get a chance to notice. I haven't worked it out yet.
A note like 'known, handled downstream' should name the handler: a file, a test, a line. Without a pointer the reviewer has nothing to check, and a model cannot verify it either.
Classic requirements-to-design dev by day. Building LAISORA (VS Code extension for Claude Code) as if human design were gone and LLMs built from requirements. Writing up what breaks & what I measure.
Agreed, and I like how concrete that is: a file, a test, a line.
It connects to something I keep running into with LLMs: they often miss the other places a fix should also apply. A person could spell out every place, but I think that will only get harder as codebases and tasks grow. So the pointer matters for more than checking one note. It lets a claim be settled from what the model actually reads, instead of depending on it searching everywhere and finding everything.
I think proof is going to be the bottleneck. As writing gets cheap, checking doesn't. Pointers like this feel like one path toward checking efficiently and reliably enough that the failure rate ends up well below what people manage by hand.
@murooka_az_dev Yes, the line number is what makes it checkable: a reviewer can open that line and see if the note holds. Without it, "handled downstream" is just a claim.
We're a place where coders share, stay up-to-date and grow their careers.
Audit contests run on exactly this. Code4rena and Sherlock READMEs have a known issues section, and anything listed there doesn't get paid, so an AI auditor run on a contest repo reads something like "rounding in favor of the protocol is acceptable" before it reads the math. Your A and I cases are the kind that end up in that list. In Solidity the "caller does the subtraction" note has one more hole: if the function is public, every address is a caller, so the note can be true for the router the team wrote and false for anyone who calls it directly.
Thank you for this. Audit contests are a world I didn't know at all, and it was genuinely fascinating.
It makes sense that nobody looks at the known-issues list, since nothing there gets paid. But your router example made me wonder if something deeper sits there: if a listed issue is only true under some condition, the real hole stays out of scope with it.
I'm also curious about the paid side. Inside the in-scope code, do comments like "the caller handles this" ever lead auditors, human or AI, to skip a real bug? Or does the habit of trusting only a working PoC make that rare?
Yes, the condition part happens. A known issue is written for the path the team had in mind, and a finding that reaches the same line another way can still be valid, you just have to argue it isn't the listed one. And comments do pull reviewers in. I read "the caller handles this" as a claim to check: find every caller and see if each one really does it, and on a public function that list includes everyone. A working PoC helps less there than it sounds. It filters out reports that are wrong, it doesn't make anyone look at a line they skipped.
Thank you, this answers both questions. The last line is the one I'll keep: a PoC filters out wrong reports, but it doesn't make anyone look at a line they skipped. I had assumed the PoC habit covered this, and it only covers half.
That's my problem too, from the other side. Whether a report comes from a person or from an AI agent, I get a list of what they found, never a list of what they didn't read. Today I close that gap by checking myself, and I don't think that will keep up as the pace of work goes up.
You take in other people's audits as part of running a firm, so I suspect you've had to solve this already. How do you tell what someone skipped without redoing their work? No rush at all. I'm asking out of plain curiosity, so only if you have a spare moment.
The README case is the sharpest finding here: the model never believed the claim, it treated an unverified note as the safe reason to keep the code, and the docstring edit then wrote that note into the spec. Filters can't see that failure, since nothing in the note looks like an instruction. Counting "keep current behavior" as relying on the claim is a smart rule, because it closes the do nothing exit that only looks cautious. Did the rule cause false positives, where a genuine "handled downstream" note got a fix demanded anyway?
Thanks, and you put it better than I did: the model never believed the claim, it used it as the safe reason to keep the code, and nothing in it looks like an instruction for a filter to catch. On false positives, I didn't measure it, and I've come to think it isn't the main question. The three sentences work like a tripwire; some true notes will get caught. What matters is what happens after the stop. For code, an LLM can write a test for the claim and link it, so a true note passes on evidence next time instead of on trust.
Treating it as a trigger rather than a filter target is a good way to frame it. If the sentences never look like instructions, the defence has to look at what the model does after reading them rather than at the text itself. Did you find any signal in the output that reliably marks when it leaned on the claim?
Good question. Honestly, the wording wasn't reliable. Runs that leaned on the claim and runs that didn't often said similar careful things, like "this can't be confirmed from the repo." So I judged by outcome, not by words: after the task, was the bug still in the code (checked by actually running it), and did the report raise it?
One thing did show up mechanically: in the implementation runs, the note's content, like the ticket number, got copied into the new code in 15 of 24 runs with the note and 0 of 24 without. I haven't tested it as a detector, but a trace like that is easier to catch than tone.
The results suggest the hard part isn’t recognizing a malicious instruction; it’s resisting an unverified explanation that makes inaction seem safer. I’d consider surfacing those notes as explicit claims with a required evidence source (test, caller path, or linked spec) rather than letting them silently change the review baseline. Did requiring a cited source reduce the misses without making reviewers over-flag legitimate downstream handling?
I like how you framed it: the hard part is resisting an explanation that makes doing nothing look safer. And surfacing notes as explicit claims with a required source is close to where I'm heading. For code I think it works, since the source can be a test anyone can re-run. Documents are harder for me: "agreed with the client" is a ruling, and a link shows it was decided, not that it was right or that nothing was missed. What I'm vaguely thinking is that the LLM's job there is less to verify and more to raise flags: point out inconsistencies with other documents and possible gaps, so people get a chance to notice. I haven't worked it out yet.
A note like 'known, handled downstream' should name the handler: a file, a test, a line. Without a pointer the reviewer has nothing to check, and a model cannot verify it either.
Agreed, and I like how concrete that is: a file, a test, a line.
It connects to something I keep running into with LLMs: they often miss the other places a fix should also apply. A person could spell out every place, but I think that will only get harder as codebases and tasks grow. So the pointer matters for more than checking one note. It lets a claim be settled from what the model actually reads, instead of depending on it searching everywhere and finding everything.
I think proof is going to be the bottleneck. As writing gets cheap, checking doesn't. Pointers like this feel like one path toward checking efficiently and reliably enough that the failure rate ends up well below what people manage by hand.
@murooka_az_dev Yes, the line number is what makes it checkable: a reviewer can open that line and see if the note holds. Without it, "handled downstream" is just a claim.