DEV Community

Cover image for Record What You Rejected, and Make Your Agent Check It: Lessons from 495 ADRs

Record What You Rejected, and Make Your Agent Check It: Lessons from 495 ADRs

Hiroki Kondo on November 10, 2026

In karasu, the architecture-diagram tool I develop, I keep design decisions as ADRs (Architecture Decision Records) under docs/adr/. I started on d...
Collapse
 
halfcycle profile image
Halfcycle •

The required step is what made it work. 495 records only help if something reads them at the moment a new idea arrives, and a step in the skill guarantees the reading happens. Recording the rejected option is underrated too: it is the one an agent is most likely to bring back, because it was reasonable enough to consider in the first place. We hit the same thing building Halfcycle: a past decision only holds if checking it is a step, not a hope. How do you keep 495 findable as they grow, by tags, or by the agent scanning titles?

Collapse
 
kompiro profile image
Hiroki Kondo • • Edited

Thanks! I do have a topic field in the frontmatter, but the session logs show the agent doesn't use it. It gets to the ADR it needs with grep.

Collapse
 
halfcycle profile image
Halfcycle •

That matches what we see too: agents search by text, not by metadata. So the words an agent would grep for belong in the title and the first line, the way a person would phrase the question. The topic field then mostly helps the humans.

Thread Thread
 
kompiro profile image
Hiroki Kondo •

Agreed on topic. In karasu it mostly serves the generated list of current ADRs, which is grouped by topic for people to browse.

On where the words go, my experiment pointed a bit wider than the title and the first line. Even when the request avoided the obvious word (asking for "a colon-and-indent notation" instead of saying "YAML"), the agent reached the ADR in all 12 trials. The rationale section uses the problem's own vocabulary (braces, indentation), so grep lands on the body too. Writing down the why also made the ADR easier to find.

Thread Thread
 
halfcycle profile image
Halfcycle •

That is a better result than my guess, and a nice one: the rationale is written in the problem's words, so it doubles as the search index. Twelve out of twelve without the obvious keyword is a good argument for writing the why at length rather than tidying it into a summary.

Collapse
 
itsvicmoreno profile image
Vic Moreno •

The GITHUB_TOKEN catch is the best case for this I've read: an Actions auto-merge would have merged fine and quietly stopped the deploys, since that push triggers no workflow. The number that worries me is the median ADR going from 2,308 to 6,941 characters, because the agent finds the rejection but has to dig for the reason. Do you split an ADR once it holds more than one decision, or keep them whole and let the index handle it?

Collapse
 
kompiro profile image
Hiroki Kondo • • Edited

Thanks for the thoughtful comment!

On the auto-merge part: if I read you right, you mean the agent turning on auto-merge. I realized I left something out of the post. I'm now moving that step to a GitHub App so it runs deterministically. That was the "new key" decision the design left to me, and I went with the App.

On ADRs getting longer: I think the main driver is the product's growth. Each decision now has more to point to as its grounds, both code and earlier ADRs.

To your question: I've kept promoting design docs to ADRs. I write one design doc per requirement, but I missed that a single design sometimes ends up with several decisions in it. So many of karasu's ADRs today hold more than one decision, and that isn't where I want to be. An ADR shouldn't change after it's written. So I'm considering a notation that says a later ADR amends only part of an earlier one (github.com/kompiro/adr-tools/issue...). If that still turns out to be hard to work with, I should probably split the decisions into separate ADRs when I promote them.

One more thing: I do generate index files, but the agent hardly uses them. Maybe the repository is still small enough that searching the ADRs directly is good enough.

Collapse
 
itsvicmoreno profile image
Vic Moreno •

The unused index makes sense at this size: grepping 495 files is cheaper for the agent than deciding to trust a summary. Where an index starts paying is the supersession map (which ADR amends or replaces which), because grep finds the old decision but not the fact that part of it is dead. Would your amends notation feed a map like that, or stay inside each ADR?

Thread Thread
 
kompiro profile image
Hiroki Kondo •

Good point. In practice, though, partial overrides are rare in karasu. I've found only two, and superseded ADRs are few as well (25 of 495). Even today you can follow an old ADR forward to its replacement and see that it's been retired. The amends notation would sit in the frontmatter of both ADRs, so it would feed the generated map too. But the session logs show the agent gets almost all of its work done with grep alone.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

A rejected option needs its reason written next to it, or the agent proposes it again as new. Without the why, the next session cannot tell a dead end from an idea nobody has tried.

Collapse
 
kompiro profile image
Hiroki Kondo •

Thanks. In my experience, the agent rarely re-proposes something I had decided not to do. I think the main reason is that the skill I design with makes it check past decisions first.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Thanks @kompiro. That is why each rejected option is stored with its reason: without the why, a dead end looks like an idea nobody has tried, and the next session proposes it again.

Collapse
 
kompiro profile image
Hiroki Kondo •

@paul_irolla_6b5ae261d0224 Agreed, and one thing I'd add: the reason has to go in when the ADR is written, because I never edit an ADR's body afterward. Only the frontmatter, the links between ADRs, gets updated. If a later decision overturns something, it becomes a new ADR that points back to the old one.

It's the same reason you don't rewrite Git history. Once editing is allowed for a good reason, the next edit comes a little easier, the policy loosens, and the ADRs stop being a record of what was known and decided at the time.

Collapse
 
pang_claw profile image
pang •

The rejection reason is the part the next session actually needs.