The L0–L3 classification is the part most "human in the loop" setups skip — treating a login-card centering and safety-critical control logic as the same review burden is why people end up rubber-stamping everything. The enforced workflow point matters too; a prompt saying "always ask" isn't a control. Do the L2/L3 plans get re-classified if the agent discovers scope creep mid-task?
Yes, and it's enforced rather than left to the agent's judgment.
Every task in an approved plan has a boundary. When the agent finds it needs to change something outside it, it files a scope change request instead of quietly continuing. Absorbing that new scope means editing the plan, and the design approval is bound to a hash of the plan, so the approval is invalidated and the state is pulled back. The human approves again before work continues.
Going back also means re-running the router. The mode can go up (L2 → L3), and it only ratchets one way for safety: a gate that was triggered by a safety flag is never released by reclassification.
There's a second outcome too. If the feature turned out to contain several results that can each be verified and shipped on their own, it stops being one feature. It's split into a group, and you approve that list before anything else is built.
So scope creep costs a round-trip to the human, which is the point: the plan you approved is the plan that gets executed.
The hash-bound approval is the detail that makes this real — invalidating the design approval on any scope edit closes the exact loophole most setups leave open. And the one-way ratchet on safety flags is the right call. Does the round-trip cost ever get painful on fast-moving work, or is that friction exactly the point?
Both, honestly. The friction is the point where it's deserved, and a bug where it isn't, so the design goal was to make it proportional rather than constant.
Every request gets graded L0–L3 before anything else. L0/L1 (center a card, rename a field) go straight to change + check with no approval at all. The design gate only applies at L2/L3, and the final gate only at L3. So fast-moving work mostly never sees a round-trip.
Where it does hurt: a mid-flight scope change on an L2/L3 feature. The hash changes, the approval drops, and the state gets pulled back to design. That's a real cost, and I've grumbled at it myself. But every time I looked at what had actually changed, it was something I'd want to have seen. The alternative is an approval that silently covers a plan I never read, which is the loophole you mentioned.
Two things keep it tolerable in practice: docs-only commits don't invalidate the final approval, and a request that's really several results gets split into separate features, so a change in one doesn't reopen the others.
The part I haven't solved is grading itself. If the tool over-grades, you pay friction for nothing, and the one-way ratchet on safety flags means it errs in that direction on purpose. I'd rather be annoyed than surprised, but I can see that trade not working for everyone.
Proportional friction is exactly the right design goal — the grading step is doing all the real work here, and it's the honest weak spot: a request graded too low never sees a gate at all. Is the grading rule-based, or does the model grade its own requests?
Split down the middle, and I know exactly what you mean, because the half that matters most is the model's.
The model does the scoring: after probing the code it gives six axes a 0–3 score (ambiguity, scope, architecture, risk, verification difficulty, dependencies) and raises flags like auth, secrets, or destructive_migration. Each score has to come with its evidence, whether it was confirmed against the code, and a confidence level.
The mapping from scores to grade is plain code, not the model: risk or architecture at 3 forces L3 on its own, any flag forces at least L2 with both gates, a safety flag forces L3, and an unconfirmed low-confidence score is bumped up by one before any of that. The model can't pick a grade, only the inputs, and it can't write the state file directly.
So your worry stands. If the model scores everything 0 with "confirmed, high confidence" and raises no flag, the rules return L0 and no gate ever appears. What exists against that today is detection, not prevention: the scores and their evidence go into a hash-chained log, so an under-grade is visible and attributable afterwards.
I don't have a finished answer for it. I use this on my own projects every day, and that's how I'm finding these holes and closing them one at a time. Under-grading is on that list, and the direction I'm leaning is a floor the model has no say in, such as paths the repo owner marks as always gated. If you have seen a setup that handles this well, I'd like to hear about it.
Review fatigue is often a classification problem before it is a staffing problem. The L0–L3 framing becomes powerful if the classification is tied to evidence: why this change is low risk, what can be auto-checked, and which condition promotes it to human review. That prevents “low risk” from becoming a label of convenience.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Red Pill Mode measures attention only as long as the planted items look like the misses that happen for real. If the plants are mostly a number that differs from the requirement in a test table, a reviewer can learn to scan tables for numbers and catch every plant without getting better at the rest, like an assumed requirement or a change outside the scope, which are the misses you describe at the start.
One way to check that the exercise transfers is to keep, next to the plant catch rate, a count of real problems found after an approval: a bug traced back to an approved change, a requirement someone notices was dropped. If the catch rate goes up and that count does not go down, the reviewer is getting better at the game, not at the review.
You're right, and I've seen it from the inside. Using it on my own projects, the plants come out too much alike, and too many of them are the number game you describe: a value in a table that differs from the requirement. After a few drills I was catching them by scanning for numbers, which is the exact failure you predicted.
I'm fixing that now. The plants are being reworked so they come from the state of the feature itself, and are harder and more plausible: a requirement that was assumed, a change outside the scope, something left out without a word. Those are the misses that happen for real, and the drill is only worth anything if it looks like them.
Your second point I agree with completely. The tool already records defects found after final approval and results the user judged different from what they meant, but they are entered by hand and reported nowhere near the catch rate, so the comparison you describe can't be made from what it prints. Those two numbers belong side by side. With one person's usage the counts will be too small to prove transfer, but a catch rate climbing while escapes stay flat would at least be visible.
Thank you for the precise version of the criticism.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
The defects your tool already records after final approval, and the results you judged different from what you meant, could also decide which plants to make. The state of the feature gives a plant its content, and the records can say which kind to plant more often. Then the drill is weighted toward the misses you actually make.
Before a record counts as an escape, I would ask one thing: was the problem visible in what you were shown when you approved? A bug that appears only after a later feature is built on top of the approved one, or a need that changed after the approval, is not a review miss, and a plant made from it could not be caught in review either. No drill can lower that part of the count, and with counts this small two or three such records can hide the effect you are looking for. Since the records are entered by hand anyway, the question can be one yes or no field.
The yes/no field is the one I can add right away. You're right that without it the count mixes two things: what I missed in review, and what nobody could have seen at approval time. With numbers this small, the second kind would hide any effect. One question at entry, "was it visible in what you were shown?", separates them.
Weighting the plants by real misses is a good next step too. Today the tool only records the way a plant was wrong so that it doesn't repeat it. Your idea is the other half: use the escape records to say which kinds to plant more often. One thing I'd watch is that the weight stays a tilt and not a rule. If my real misses are mostly missing checks and every drill becomes a missing check, I'm back to a pattern the reviewer can learn.
I'll put both in an issue on the repo. If you try it and anything else looks off, please let me know.
The Red Pill Mode point is the one that sticks with me — an approval gate only works if the human is actually reading, and most systems never test that. The L0–L3 grading is what makes it workable in practice, though: asking for the same level of review on a login-card tweak as on safety-critical logic is exactly how reviewers learn to click approve without looking.
Thanks, that's exactly the pair I was going for. The two parts need each other: grading keeps the number of approval requests low enough that reading each one is realistic, and Red Pill Mode checks that the reading really happens.
The weak spot is the grading itself. If the model scores a risky change too low, nobody is asked at all. Today that is only visible afterwards in the log, so the next two things I'm building are a floor (paths the owner marks as always gated) and a small random sample of ungated changes: github.com/kiduki77/human-rights/i... github.com/kiduki77/human-rights/i...
If you try it and anything looks off, please let me know.
hash-bound approval is the part I'd borrow. a scope change keeping the old approval is the quiet failure. how do you keep L0-L3 from over-grading a small fix into L2?
Ha, the bit where it mocks you for approving blind is going to land with a lot of people. Do you reckon reviewers just learn to spot the planted ones, like phishing test emails, and skim the rest?
Yes, that's the failure mode I'm designing against. Phishing tests get easy because the fakes share a look. Mine started to as well: mostly a number that didn't match the requirement, and I was catching them by scanning for numbers.
So the drill now adapts on two axes. Difficulty goes up after clean catches and down after misses. And the way a plant is wrong is recorded, so the next drills use a different one: a check that quietly went missing, an assumption nobody agreed to, a decision credited to the wrong party, a test moved to a requirement it doesn't verify. A number-only change is allowed once per set and never at the hard level.
The hard ones are a real row changed or removed, never an extra row, so nothing in the row looks odd by itself. You only find it by comparing with the requirements, which is the same work as reviewing the real rows. If there's no pattern to learn, skimming stops paying off. Whether that carries over to real misses is what I still need to measure. :)
We're a place where coders share, stay up-to-date and grow their careers.
The L0–L3 classification is the part most "human in the loop" setups skip — treating a login-card centering and safety-critical control logic as the same review burden is why people end up rubber-stamping everything. The enforced workflow point matters too; a prompt saying "always ask" isn't a control. Do the L2/L3 plans get re-classified if the agent discovers scope creep mid-task?
Yes, and it's enforced rather than left to the agent's judgment.
Every task in an approved plan has a boundary. When the agent finds it needs to change something outside it, it files a scope change request instead of quietly continuing. Absorbing that new scope means editing the plan, and the design approval is bound to a hash of the plan, so the approval is invalidated and the state is pulled back. The human approves again before work continues.
Going back also means re-running the router. The mode can go up (L2 → L3), and it only ratchets one way for safety: a gate that was triggered by a safety flag is never released by reclassification.
There's a second outcome too. If the feature turned out to contain several results that can each be verified and shipped on their own, it stops being one feature. It's split into a group, and you approve that list before anything else is built.
So scope creep costs a round-trip to the human, which is the point: the plan you approved is the plan that gets executed.
Thank you for asking :)
The hash-bound approval is the detail that makes this real — invalidating the design approval on any scope edit closes the exact loophole most setups leave open. And the one-way ratchet on safety flags is the right call. Does the round-trip cost ever get painful on fast-moving work, or is that friction exactly the point?
Both, honestly. The friction is the point where it's deserved, and a bug where it isn't, so the design goal was to make it proportional rather than constant.
Every request gets graded L0–L3 before anything else. L0/L1 (center a card, rename a field) go straight to change + check with no approval at all. The design gate only applies at L2/L3, and the final gate only at L3. So fast-moving work mostly never sees a round-trip.
Where it does hurt: a mid-flight scope change on an L2/L3 feature. The hash changes, the approval drops, and the state gets pulled back to design. That's a real cost, and I've grumbled at it myself. But every time I looked at what had actually changed, it was something I'd want to have seen. The alternative is an approval that silently covers a plan I never read, which is the loophole you mentioned.
Two things keep it tolerable in practice: docs-only commits don't invalidate the final approval, and a request that's really several results gets split into separate features, so a change in one doesn't reopen the others.
The part I haven't solved is grading itself. If the tool over-grades, you pay friction for nothing, and the one-way ratchet on safety flags means it errs in that direction on purpose. I'd rather be annoyed than surprised, but I can see that trade not working for everyone.
Proportional friction is exactly the right design goal — the grading step is doing all the real work here, and it's the honest weak spot: a request graded too low never sees a gate at all. Is the grading rule-based, or does the model grade its own requests?
Split down the middle, and I know exactly what you mean, because the half that matters most is the model's.
The model does the scoring: after probing the code it gives six axes a 0–3 score (ambiguity, scope, architecture, risk, verification difficulty, dependencies) and raises flags like auth, secrets, or destructive_migration. Each score has to come with its evidence, whether it was confirmed against the code, and a confidence level.
The mapping from scores to grade is plain code, not the model: risk or architecture at 3 forces L3 on its own, any flag forces at least L2 with both gates, a safety flag forces L3, and an unconfirmed low-confidence score is bumped up by one before any of that. The model can't pick a grade, only the inputs, and it can't write the state file directly.
So your worry stands. If the model scores everything 0 with "confirmed, high confidence" and raises no flag, the rules return L0 and no gate ever appears. What exists against that today is detection, not prevention: the scores and their evidence go into a hash-chained log, so an under-grade is visible and attributable afterwards.
I don't have a finished answer for it. I use this on my own projects every day, and that's how I'm finding these holes and closing them one at a time. Under-grading is on that list, and the direction I'm leaning is a floor the model has no say in, such as paths the repo owner marks as always gated. If you have seen a setup that handles this well, I'd like to hear about it.
Review fatigue is often a classification problem before it is a staffing problem. The L0–L3 framing becomes powerful if the classification is tied to evidence: why this change is low risk, what can be auto-checked, and which condition promotes it to human review. That prevents “low risk” from becoming a label of convenience.
Red Pill Mode measures attention only as long as the planted items look like the misses that happen for real. If the plants are mostly a number that differs from the requirement in a test table, a reviewer can learn to scan tables for numbers and catch every plant without getting better at the rest, like an assumed requirement or a change outside the scope, which are the misses you describe at the start.
One way to check that the exercise transfers is to keep, next to the plant catch rate, a count of real problems found after an approval: a bug traced back to an approved change, a requirement someone notices was dropped. If the catch rate goes up and that count does not go down, the reviewer is getting better at the game, not at the review.
You're right, and I've seen it from the inside. Using it on my own projects, the plants come out too much alike, and too many of them are the number game you describe: a value in a table that differs from the requirement. After a few drills I was catching them by scanning for numbers, which is the exact failure you predicted.
I'm fixing that now. The plants are being reworked so they come from the state of the feature itself, and are harder and more plausible: a requirement that was assumed, a change outside the scope, something left out without a word. Those are the misses that happen for real, and the drill is only worth anything if it looks like them.
Your second point I agree with completely. The tool already records defects found after final approval and results the user judged different from what they meant, but they are entered by hand and reported nowhere near the catch rate, so the comparison you describe can't be made from what it prints. Those two numbers belong side by side. With one person's usage the counts will be too small to prove transfer, but a catch rate climbing while escapes stay flat would at least be visible.
Thank you for the precise version of the criticism.
The defects your tool already records after final approval, and the results you judged different from what you meant, could also decide which plants to make. The state of the feature gives a plant its content, and the records can say which kind to plant more often. Then the drill is weighted toward the misses you actually make.
Before a record counts as an escape, I would ask one thing: was the problem visible in what you were shown when you approved? A bug that appears only after a later feature is built on top of the approved one, or a need that changed after the approval, is not a review miss, and a plant made from it could not be caught in review either. No drill can lower that part of the count, and with counts this small two or three such records can hide the effect you are looking for. Since the records are entered by hand anyway, the question can be one yes or no field.
Both points are useful, thank you.
The yes/no field is the one I can add right away. You're right that without it the count mixes two things: what I missed in review, and what nobody could have seen at approval time. With numbers this small, the second kind would hide any effect. One question at entry, "was it visible in what you were shown?", separates them.
Weighting the plants by real misses is a good next step too. Today the tool only records the way a plant was wrong so that it doesn't repeat it. Your idea is the other half: use the escape records to say which kinds to plant more often. One thing I'd watch is that the weight stays a tilt and not a rule. If my real misses are mostly missing checks and every drill becomes a missing check, I'm back to a pattern the reviewer can learn.
I'll put both in an issue on the repo. If you try it and anything else looks off, please let me know.
The Red Pill Mode point is the one that sticks with me — an approval gate only works if the human is actually reading, and most systems never test that. The L0–L3 grading is what makes it workable in practice, though: asking for the same level of review on a login-card tweak as on safety-critical logic is exactly how reviewers learn to click approve without looking.
Thanks, that's exactly the pair I was going for. The two parts need each other: grading keeps the number of approval requests low enough that reading each one is realistic, and Red Pill Mode checks that the reading really happens.
The weak spot is the grading itself. If the model scores a risky change too low, nobody is asked at all. Today that is only visible afterwards in the log, so the next two things I'm building are a floor (paths the owner marks as always gated) and a small random sample of ungated changes:
github.com/kiduki77/human-rights/i...
github.com/kiduki77/human-rights/i...
If you try it and anything looks off, please let me know.
hash-bound approval is the part I'd borrow. a scope change keeping the old approval is the quiet failure. how do you keep L0-L3 from over-grading a small fix into L2?
Ha, the bit where it mocks you for approving blind is going to land with a lot of people. Do you reckon reviewers just learn to spot the planted ones, like phishing test emails, and skim the rest?
Yes, that's the failure mode I'm designing against. Phishing tests get easy because the fakes share a look. Mine started to as well: mostly a number that didn't match the requirement, and I was catching them by scanning for numbers.
So the drill now adapts on two axes. Difficulty goes up after clean catches and down after misses. And the way a plant is wrong is recorded, so the next drills use a different one: a check that quietly went missing, an assumption nobody agreed to, a decision credited to the wrong party, a test moved to a requirement it doesn't verify. A number-only change is allowed once per set and never at the hard level.
The hard ones are a real row changed or removed, never an extra row, so nothing in the row looks odd by itself. You only find it by comparing with the requirements, which is the same work as reviewing the real rows. If there's no pattern to learn, skimming stops paying off. Whether that carries over to real misses is what I still need to measure. :)