DEV Community

Heinrich Neb
Heinrich Neb

Posted on

My AI remembered three true things. Each one still broke something.

In the last part I asked what your AI had remembered wrong. Sixteen of you answered with real cases, and I expected a pile of stale notes. Most of the cases taught me something else: the notes were true, and they still sent the agent the wrong way.

Your cases sorted themselves into three kinds. Each kind needs its own fix, which also explains why a single "freshness" score never worked for me.

1. True once, then replaced

This is where I started. My agent kept quoting "four to one" after I had measured "two to one".

@paullovelace had the same with a renamed API endpoint. The correction happened in chat, and for weeks the agent used the old name, because the correction never reached the memory file. Paul's case sits one step earlier than where I had been looking. Before a correction can win, it has to arrive in the store at all.

Then it has to win. Mine lets a new note name the one it replaces, so the old one stays readable but stops winning. When I counted, 2 of 735 notes carried that link. The mechanism works; agents just rarely use it.

@gimalay (Dmytro) does it differently, and I think that's right. In Dmytro's tool, a note marked as replaced can't be saved without naming its replacement, and sessions load only current notes. Asking nicely gave me 2 of 735, so I'll try it that way: the question becomes part of the write.

@opschainx goes further and deletes the old note outright, and named an honest gap in that, too: the same fact relearned in different words survives the delete. zhang jimmy and @salparvez landed in between, with in-place replacement and a "void" stamp. Side by side, it made sense to me: the old note should stay as history for a human and go quiet for the agent.

2. True for its case, used for another

@galadev had a case I hadn't thought about. Gala's agent keeps a file of corrections. One said never to link a certain subdomain, because that app has its own domain. Eight days later the agent applied the rule to a different site that has no domain of its own. The note was right. It had lost the case it came from.

A replace link can't help, because nothing replaced the note. What helped was writing the scope into the note: which site, which service, which environment. Gala found it by asking the agent where the rule came from, and the pointer led to the wrong site. I'll do that too. A rule that can say where it came from can also say where it stops.

3. True for its data, read without it

@izgorodin (Edward) pointed out that my own two numbers were both honest. "Four to one" and "two to one" were each true of the data they were computed on; the second just rested on more. I think Edward is right: stale facts need a replacement link, stale measurements mostly need their denominator.

So a number now goes into memory with its window and its count next to it: "two to one, N articles, June to August". The agent can then see that a new reading is due once the site has grown, instead of choosing between two equally sure figures.

Two lessons that cut across all three

@kitwisestudio described the workaround for a bug that has since been fixed. It never looks stale, and it keeps getting written back into new code. I checked my own store: it can attach a version range to a note, and recall pushes the note down once the project is past the range. It only helps when someone writes the range at the time, the same weak spot as the link.

@maame-codes, @quantumtechcoder and @halfcycle arrived at the same rule from three directions: for anything that can change, read the live source first. Ports, IDs, versions, config. QuantumTechCoder's migrations ran cleanly against a dead database for 25 minutes; Halfcycle's agents kept hedging about features that had shipped. Memory brings the context; the file on disk brings the value.

What I'm changing

  • The "does this replace an older note?" question becomes part of the write, not something the agent may skip.
  • Notes get room for their scope, so a rule knows where it stops.
  • Numbers get their window and their count.
  • Corrections given in chat get caught and offered for storage, so they reach the store at all.

I'll report the replace-link count again in a few weeks. 2 of 735 is the baseline.

Thank you also to Ayush Dhanani, OneLifeVibe, Kevin Hall, Mudassir Khan and everyone else who added a case. Every case helped.

Which of the three have you run into most: the replaced note, the note used outside its case, or the number without its denominator?


I build cachly. It keeps what you explain to your AI: your rules, your decisions and the bugs you already solved. Claude Code, Cursor, Copilot and the others read it in every session, for you alone or for your whole team, and every lesson carries the name of whoever learned it.

Try it, in increasing order of commitment:

  • 30 seconds, no account: npx @cachly-dev/mcp-server@latest demo in any git repo. It reads your log locally and prints what an assistant would already know about the project. After npx fetches the package, the command makes no network calls.
  • Claude Code plugin: /plugin marketplace add cachly-dev/cachly-mcp, then /plugin install cachly-brain@cachly.
  • 5 minutes, free tier: npx @cachly-dev/mcp-server@latest autopilot writes the MCP configuration for whichever assistant you use.
  • Or from the web: cachly.dev · free tier, servers in Germany.

Top comments (7)

Collapse
 
halfcycle profile image
Halfcycle •

Case 2 is the one we hit most: a decision recorded without its scope gets applied everywhere. "A rule that can say where it came from can also say where it stops" is the fix we landed on too. Have you tried making the agent name the note it acted on, so a wrong application shows up in review instead of eight days later?

Collapse
 
heinrichneb profile image
Heinrich Neb •

Not as a rule yet. Each recalled note already comes back with its name, so the agent can say which one it used, but nothing makes it do that. Your version is cheaper than mine: the citation lands in the diff or the PR text, where someone reads it anyway. I'll try it as one line in the agent instructions first and count how often a wrong application gets caught there.

Collapse
 
izgorodin profile image
Edward Izgorodin •

Your 2 of 735 is the third case in your own list: a number without the denominator it needs. Most of the 735 notes replace nothing, so they cannot carry a link in the first place. The count that matters is the notes that do replace an older one. A hand check of 50 random notes gives a rough share, and if a tenth of them replace something, the baseline is 2 of about 74, not 2 of 735. New notes will keep growing 735 whether or not they replace anything, so when you report again in a few weeks, both counts side by side will show whether the write-time question works. The third kind is also the one I run into most.

Collapse
 
heinrichneb profile image
Heinrich Neb •

Fair, and it is my own third case. 2 of 735 mixes notes that could never carry a link with the ones that should. I'll hand-check a random 50 for the share that replaces something and report both counts side by side next time, so the denominator is visible instead of implied.

Collapse
 
gautamtalksdev profile image
Gautam K •

"Three true things that each broke something" is a great finding. True but stale memory is worse than no memory, because the agent acts on it confidently. Expiry dates on remembered facts would help a lot.

Collapse
 
quantumtechcoder profile image
QuantumTechCoder •

Nice work! This one actually aims at solving the problem most of us faced for a long time with AI. Will try it! Also thanks for mentioning me in the post :D

Collapse
 
heinrichneb profile image
Heinrich Neb •

Your D1 binding case was the clearest example of the replaced note, so it had to be in there.

If you try it: the quickest path is npx @cachly-dev/mcp-server setup. If the sign-in step gets in your way, tell me here. That step is what I'm fixing this week.