Here’s a scenario that’s become painfully common.
You catch your AI agent making the same mistake for the third time. You tell it to save the correction. It writes a rule to its memory file. You watch it happen. The file updates on disk. The agent confirms: “Saved! I’ll remember this.”
Next session. Same mistake.
You’re not crazy. The agent really did write the file. The file really does exist on disk. And it really will ignore it completely. This is the #1 failure mode in LLM agent systems, and it’s almost never that the agent “didn’t save.” It’s that saving and learning are two entirely different operations.
The memory illusion
People assume: if the agent wrote it down, it learned it.
In reality: the agent recorded information. Whether that information influences future decisions depends on whether the agent retrieves it at the right time, in the right format, at the right position in its prompt. Most of the time, it doesn’t.
Think of it like this. You write “never touch the production database” in a notebook. If you never open the notebook again, you’ll still touch the production database. The notebook isn’t the problem. The retrieval is.
An agent’s memory file is just the notebook.
Seven reasons the notebook doesn’t work
After digging into this across multiple agent frameworks — Hermes, Claude Code, OpenHands, and others — the same patterns keep showing up. There are seven distinct failure modes, and most agents hit at least three.
1. The snapshot freeze
Most agent frameworks load memory files as a frozen snapshot at session start. This is a performance optimization — it preserves LLM prefix caching so the model doesn’t re-process the same system prompt every turn.
The catch: when the agent writes a correction mid-session, the file updates on disk, but the active system prompt stays frozen. The agent literally cannot see its updated memory for the rest of that session. It will keep making the same mistake until you start a new one.
This is the most common cause. It’s also the easiest to test: does the mistake stop after a fresh session? If yes, that’s your answer.
2. The wrong file
Agent memory systems typically have multiple tiers. In Hermes, for example:
- SOUL.md — permanent identity rules, only the human edits
- USER.md — user preferences, agent can update
- MEMORY.md — working notes, subject to automatic summarization
The trap: if the agent writes your correction to MEMORY.md, it gets summarized and consolidated over time. The nuance gets compressed into something vague. Or it gets evicted entirely when the file hits its character limit (around 2,200 characters by default).
Meanwhile, MEMORY.md is recalled via full-text search. If your future query doesn’t match the keywords in the saved note, it won’t be pulled in. The lesson sits on disk, invisible.
Hard rules that must never be violated belong in SOUL.md or AGENTS.md. Putting them in MEMORY.md is like putting a restraining order in a sticky note.
3. Skills are opt-in
In most agent systems, skills or learned procedures aren’t auto-executed. At session start, the agent gets a compact list of skill names and short descriptions. It must voluntarily call a load function to read the full content.
Weaker models frequently ignore the “MUST load matching skills” instruction. Stronger models skip it sometimes too. There’s no automatic trigger matching that forces a skill to load based on context.
Even when a skill exists, if its trigger description doesn’t precisely match how the mistake re-manifests — different wording, different tool call sequence, different task framing — it never fires.
4. The character limit
Memory files are small. MEMORY.md: ~2,200 characters. USER.md: ~1,375. When they fill up, the agent consolidates older entries to make room. Your specific correction might get paraphrased into something generic, or dropped entirely in favor of “more important” entries.
You can increase these limits (hermes config set memory.memory_char_limit 5000), but the fundamental problem remains: a bounded text file managed by an LLM will eventually compress away the details that matter.
5. Lost in the middle
Even when memory IS injected into the prompt, LLMs have a well-documented attention problem. Research from Stanford (arXiv 2307.03172) showed that language models under-attend to information stuck in the middle of long contexts. They remember the beginning and the end. The middle gets ignored.
If your agent loads 400 lines of skills and the critical rule is on line 200, there’s a real chance the model skips it. Prompt ordering matters more than people realize:
Bad: System prompt → User request → Files → History → Memory
Good: System prompt → Critical rules → Task → Files
The rules should be first. Most frameworks put them last.
6. Compression failures with reasoning models
If you’re running a reasoning model (DeepSeek-R1, Qwen reasoning variants) as your main or auxiliary model, there’s an additional landmine. Agent frameworks compress long conversations to stay within context limits. Reasoning models require passing back prior reasoning content for API calls — but the compression workflow doesn’t support this.
After failed compression attempts, some frameworks trigger an auto-reset that wipes the entire session context. The agent forgets everything, including persisted rules. Mid-conversation. Without warning.
7. Sub-agents start from zero
When an agent spawns a sub-agent for a delegated task, the sub-agent launches with a completely fresh context. No inherited memory. No skills. No conversation history. If you don’t explicitly pack the rules into the task description, the sub-agent will repeat the exact same mistake the parent just learned from.
This is the one that bites people who think they’ve built a sophisticated multi-agent system. The sophistication is an illusion if the context doesn’t transfer.
Memory isn’t intelligence
The core problem is conceptual. People treat memory as synonymous with learning. It isn’t.
A human who writes “always validate input” in a notebook hasn’t learned to validate input. They’ve recorded an intention. The learning happens when they encounter a situation where input validation matters, recall the rule, and actually apply it. The recall step is where most agents fail.
LLMs are probability machines. If the prompt is almost identical to a previous one where the model chose a wrong action, it will choose that same action again — unless something in the context significantly shifts the probability distribution. A memory file sitting somewhere in the prompt, possibly buried in the middle, possibly summarized into vagueness, doesn’t shift the distribution enough.
The model doesn’t “know” it made a mistake. It doesn’t have a concept of “I did this before and it was wrong.” It has tokens in a context window and a probability distribution over next tokens. If the tokens from the memory file aren’t salient enough relative to the task tokens, the task pattern wins.
What actually works
The fixes, ranked by impact:
1. Restart the session after saving
This alone fixes a huge percentage of cases. Memory files are snapshots. Changes don’t take effect until the next session load. If you told the agent to save a rule and kept chatting in the same session, the rule was never active.
2. Put hard rules in AGENTS.md or SOUL.md
AGENTS.md is read at session start, every session, automatically. It’s not subject to summarization or character limits (within reason). Put your “never do X” constraints here, not in skills or MEMORY.md.
Write them as imperative, specific rules:
- Bad: “Be careful with database queries”
- Good: “NEVER run DELETE or DROP queries without explicit user confirmation. Always wrap writes in transactions.”
3. Force the skill by name
Don’t rely on the agent to figure out which skill applies. Tell it explicitly: “Use the deploy-staging skill.” This bypasses the voluntary load step entirely.
4. Write memories as conditional rules, not diary entries
Bad: “I noticed that when I parse JSON, I forget brackets.”
Good: “IF parsing JSON output THEN always wrap in brackets [] before sending to API.”
Even better: Store the exact failed action alongside the fix:
FAILED: Called api/v1/users → stale data
FIXED: Always call api/v2/users → current dataLLMs learn better from negative examples than positive advice.
5. Add a verification gate
Don’t let the agent declare success without proving it. Force it to run a check after completing the task:
After task completion, run: grep -r "bad_pattern"./ || echo "PASS"
If FAIL, re-apply correction.This turns a vague rule into an executable checkpoint. Advice is optional. A verification command is not.
6. For sub-agents: pack rules into context
When delegating tasks, put the critical rules directly in the task description:
CRITICAL: Never use api/v1. Always api/v2.
Last attempt used v1 and got stale data.
File: /path/to/file.pyBeing lazy about the context field is the #1 reason sub-agent tasks fail.
The golden insight
Verification and retrieval are more important than storage.
The biggest improvement doesn’t come from adding more memory. It comes from making the agent retrieve the right memory before planning and prove to itself that the mistake has actually been fixed before it finishes.
The failure loop: Task → Plan → Execute → Fail → Save to memory → Never retrieve → Fail again.
The success loop: Task → Retrieve relevant rules → Plan around them → Execute → Verify → Done.
Most agent architectures do the first loop and wonder why the agent keeps repeating itself. The fix isn’t more memory. It’s better plumbing between the memory and the decision.
The bottom line
AI agents don’t learn from files. They learn from context. A file only matters if its contents end up in the right place in the prompt, at the right time, formatted in a way the model can act on. Most of the time, they don’t.
If your agent keeps repeating the same mistake, don’t add another note to the memory file. Check whether the existing notes are actually being read. Check whether they’re in the right file. Check whether the session is stale. Check whether the formatting is actionable.
The notebook isn’t the problem. The retrieval is.
Sources: