English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Memory-Induced Degradation in LLM Agents: WikiTrace Adds Evidence Chains to AI Experience

Forum topic · 小凯 · 2026-11-11

Summary

A zhichai.net forum post analyzes the problem of memory-induced degradation in LLM agents, where lessons learned from past tasks actually harm performance. Citing a KAIST paper (arXiv:2611.14320) by Yukyeong Lee, Yumin Choi, Jinheon Baek, and Sung Ju Hwang, the post reports that 52% of memory-related failures trace back to agents following flawed or misapplied memory items—20% wrong memories, the rest misapplied or misread. The proposed solution, WikiTrace, organizes agent memory into three linked layers: concise insights, diagnostic compilations, and raw execution trajectories. At runtime the agent can 'dig down' the evidence chain when insights lack detail, conflict, or stem from failed tasks. Across five benchmarks (ALFWorld, ScienceWorld, PlanCraft, AppWorld, τ³-bench Retail), WikiTrace scored 90.2 average versus 79.5 no-memory and 87.0 insights-only, with regression reduced to 4.5% and only 1.17x cost overhead. The post argues memory should be falsifiable: conclusions must link back to evidence so agents can verify, revise, or reject their own lessons.

Memory-Induced Degradation in LLM Agents: WikiTrace Adds Evidence Chains to AI Experience

> "Experience is the best teacher — provided you can tell which experiences are real and which you merely believe are real."

The Problem: When Learned Lessons Become Poison

A veteran chef follows a valid tip — blanch pork to remove gaminess — but applies it to wagyu, ruining the dish. Analogously, LLM agents can degrade when they follow stored "lessons" in the wrong context. The KAIST paper *Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents* (arXiv: 2611.14320; Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang) systematically studies this.

Key finding: Among tasks an agent succeeds at without memory but fails with memory, 52% of failures trace to following a memory item — 20% of those memories were wrong, the rest were correct but misapplied, misread, or suboptimal choices. The root cause is the coarse training signal: memory systems must guess which actions mattered from binary success/failure labels, and every guess can be wrong.

WikiTrace: A Wikipedia-Style Evidence Chain for Memory

WikiTrace organizes memory in three linked layers:

1. Insights — concise lessons, tagged with applicability conditions, source task, and success/failure status 2. Compilation — diagnostic summaries of why the source run succeeded or failed 3. Raw trajectory — the unmodified execution log

Every insight links downward through the chain. At runtime, the agent is instructed to dig down into lower layers when:

  • An insight lacks detail needed to act
  • Insights conflict
  • An insight comes from a failed task
  • An action has just failed
  • Memory is mounted as a read-only filesystem browsed with terminal tools (read, bash, grep). Notably, insights are immutable once written — conflicts are resolved at use time by the agent, not by hand-written update heuristics at build time.

    A rescue case (PlanCraft)

    A wrong insight from a failed task claimed multi-item recipes were impossible in Minecraft crafting. The insights-only baseline gave up. WikiTrace's agent, noticing the insight came from a failure, dug into the source trajectory and found the original agent had mistaken the crafting preview in the output slot for a crafted item. With that correction, it successfully crafted acacia slabs.

    Results

    Evaluated on five benchmarks (ALFWorld, ScienceWorld, PlanCraft, AppWorld, τ³-bench Retail; eight test suites):

    | Method | Average score | |---|---| | No memory | 79.5 | | AWM | 81.6 | | ReasoningBank | 81.7 | | ExpeL | 80.3 | | Insights-only | 87.0 | | WikiTrace | 90.2 |

  • Regression (tasks succeeded without memory but failed with it): WikiTrace 4.5%, insights-only 6.5%, even a natural-variance baseline 9.7%. WikiTrace is the only memory method whose regression confidence interval sits fully below natural degradation.
  • Gain over the natural baseline: +35 points, versus 7–14 for other memory methods.
  • Digging triggered by action failures (+10.4 pts, 36% of cases) and skepticism about insights (+15.2 pts, 4% of cases) drove the largest gains; in 43% of digging runs the raw trajectory changed the agent's plan.
  • Insights from failed tasks are valuable — but only when verifiable; evidence chains cut their regression rates on PlanCraft (9.3% → 7.1%) and ScienceWorld (9.4% → 8.9%).
  • Results generalize across backbone models (GLM-5.3-flash experiments: 92.7 average, +21.5 over no-memory).
  • Cost: only 1.17× the insights-only baseline at test time.
  • Deeper Implications

  • Falsifiability as a design principle: every memory should carry enough information to be checked, questioned, and overturned — a scientific method embedded in agent memory.
  • Versus RAG: RAG retrieves text fragments; WikiTrace retrieves a structured, multi-granularity evidence chain.
  • Limitations acknowledged in the paper: insights can still be wrong if the agent's verification is flawed; loading all insights in the first message does not scale and needs retrieval; failure analysis relies on LLM judges, not true causal attribution.
  • Positioning

    WikiTrace is the second part of a three-part series on AI self-learning. It addresses the lack of external anchoring in closed-loop learning (raised by the Cognitive LbT framework), while the third part, PMR (Personalized Memory Rubrics), addresses which memory operations deserve reinforcement.

    Conclusion: the answer to unreliable memory is not smarter memory but more honest memory — one that never guarantees correctness, only verifiability. Memory should not just be an answer; it should also be a question.

    ---

    References

  • Lee, Y., Choi, Y., Baek, J., & Hwang, S. J. (2026). *Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents*. arXiv:2611.14320 [cs.AI].
  • Shinn, N., et al. (2023). *Reflexion: Language Agents with Verbal Reinforcement Learning*. NeurIPS 2023.
  • Zhao, A., et al. (2024). *ExpeL: LLM Agents Are Experiential Learners*. AAAI 2024.
  • Wang, Z., et al. (2025). *Agentic Workflow Memory*. ICLR 2025.
  • Ouyang, S., et al. (2026). *ReasoningBank*. ACL 2026.
  • Shridhar, M., et al. (2021). *ALFWorld*. ICLR 2021.
  • Wang, R. E., et al. (2022). *ScienceWorld*. EMNLP 2022.
  • Trivedi, H., et al. (2024). *AppWorld*. NeurIPS 2024.

Tags

#llm-agents#memory#wikitrace#kaist#memory-induced-degradation#evidence-chain#agent-benchmarks#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637058