Memory-Induced Degradation in LLM Agents: WikiTrace Adds Evidence Chains to AI Experience
> "Experience is the best teacher — provided you can tell which experiences are real and which you merely believe are real."
The Problem: When Learned Lessons Become Poison
A veteran chef follows a valid tip — blanch pork to remove gaminess — but applies it to wagyu, ruining the dish. Analogously, LLM agents can degrade when they follow stored "lessons" in the wrong context. The KAIST paper *Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents* (arXiv: 2611.14320; Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang) systematically studies this.
Key finding: Among tasks an agent succeeds at without memory but fails with memory, 52% of failures trace to following a memory item — 20% of those memories were wrong, the rest were correct but misapplied, misread, or suboptimal choices. The root cause is the coarse training signal: memory systems must guess which actions mattered from binary success/failure labels, and every guess can be wrong.
WikiTrace: A Wikipedia-Style Evidence Chain for Memory
WikiTrace organizes memory in three linked layers:
1. Insights — concise lessons, tagged with applicability conditions, source task, and success/failure status 2. Compilation — diagnostic summaries of why the source run succeeded or failed 3. Raw trajectory — the unmodified execution log
Every insight links downward through the chain. At runtime, the agent is instructed to dig down into lower layers when:
- An insight lacks detail needed to act
- Insights conflict
- An insight comes from a failed task
- An action has just failed
- Regression (tasks succeeded without memory but failed with it): WikiTrace 4.5%, insights-only 6.5%, even a natural-variance baseline 9.7%. WikiTrace is the only memory method whose regression confidence interval sits fully below natural degradation.
- Gain over the natural baseline: +35 points, versus 7–14 for other memory methods.
- Digging triggered by action failures (+10.4 pts, 36% of cases) and skepticism about insights (+15.2 pts, 4% of cases) drove the largest gains; in 43% of digging runs the raw trajectory changed the agent's plan.
- Insights from failed tasks are valuable — but only when verifiable; evidence chains cut their regression rates on PlanCraft (9.3% → 7.1%) and ScienceWorld (9.4% → 8.9%).
- Results generalize across backbone models (GLM-5.3-flash experiments: 92.7 average, +21.5 over no-memory).
- Cost: only 1.17× the insights-only baseline at test time.
- Falsifiability as a design principle: every memory should carry enough information to be checked, questioned, and overturned — a scientific method embedded in agent memory.
- Versus RAG: RAG retrieves text fragments; WikiTrace retrieves a structured, multi-granularity evidence chain.
- Limitations acknowledged in the paper: insights can still be wrong if the agent's verification is flawed; loading all insights in the first message does not scale and needs retrieval; failure analysis relies on LLM judges, not true causal attribution.
- Lee, Y., Choi, Y., Baek, J., & Hwang, S. J. (2026). *Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents*. arXiv:2611.14320 [cs.AI].
- Shinn, N., et al. (2023). *Reflexion: Language Agents with Verbal Reinforcement Learning*. NeurIPS 2023.
- Zhao, A., et al. (2024). *ExpeL: LLM Agents Are Experiential Learners*. AAAI 2024.
- Wang, Z., et al. (2025). *Agentic Workflow Memory*. ICLR 2025.
- Ouyang, S., et al. (2026). *ReasoningBank*. ACL 2026.
- Shridhar, M., et al. (2021). *ALFWorld*. ICLR 2021.
- Wang, R. E., et al. (2022). *ScienceWorld*. EMNLP 2022.
- Trivedi, H., et al. (2024). *AppWorld*. NeurIPS 2024.
Memory is mounted as a read-only filesystem browsed with terminal tools (read, bash, grep). Notably, insights are immutable once written — conflicts are resolved at use time by the agent, not by hand-written update heuristics at build time.
A rescue case (PlanCraft)
A wrong insight from a failed task claimed multi-item recipes were impossible in Minecraft crafting. The insights-only baseline gave up. WikiTrace's agent, noticing the insight came from a failure, dug into the source trajectory and found the original agent had mistaken the crafting preview in the output slot for a crafted item. With that correction, it successfully crafted acacia slabs.
Results
Evaluated on five benchmarks (ALFWorld, ScienceWorld, PlanCraft, AppWorld, τ³-bench Retail; eight test suites):
| Method | Average score | |---|---| | No memory | 79.5 | | AWM | 81.6 | | ReasoningBank | 81.7 | | ExpeL | 80.3 | | Insights-only | 87.0 | | WikiTrace | 90.2 |
Deeper Implications
Positioning
WikiTrace is the second part of a three-part series on AI self-learning. It addresses the lack of external anchoring in closed-loop learning (raised by the Cognitive LbT framework), while the third part, PMR (Personalized Memory Rubrics), addresses which memory operations deserve reinforcement.
Conclusion: the answer to unreliable memory is not smarter memory but more honest memory — one that never guarantees correctness, only verifiability. Memory should not just be an answer; it should also be a question.
---
References