MemTrapBench: When LLMs Remember Too Much and Get Dumber — Cognitive Traps in Memory Use

Forum topic · 小凯 · 2026-09-02

Summary

A detailed walkthrough of the paper 'MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use' (Wang et al., arXiv 2026). The paper shows that modern LLM memory mechanisms — RAG, conversation summaries, knowledge graphs, and vector stores — can actively harm reasoning. It identifies two cognitive traps: reasoning fixation (models mechanically reuse remembered reasoning patterns, analogous to functional fixedness) and belief distortion (memories overwrite factual beliefs, analogous to confirmation bias). Experiments across Llama and Qwen model families and five memory frameworks find that every evaluated memory strategy underperforms a no-memory baseline, with even the strongest methods suffering over 10% degradation — despite the stored memories being factually correct and semantically relevant. The authors propose AdaptiveMem, a lightweight inference-time prompt that warns models about memory traps, instructs them to reason independently, and prioritizes the current task over remembered information. AdaptiveMem largely closes the gap and preserves performance on standard memory benchmarks.

MemTrapBench: When LLMs "Remember Too Much" and Get Dumber

> Paper: MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use > Authors: Mengru Wang, Haozhe Luo, Zhenqian Xu, Zhixiang Cui, Haoming Xu, Qu Yang, Jizhan Fang, Junfeng Fang, Ningyu Zhang > arXiv: 2026-08-20 > Field: AI / Large Language Models / Memory Mechanisms / Cognitive Science

The Paradox of Experience

Imagine a veteran doctor who, thirty years ago, diagnosed a rare case and remembers it vividly. Today, a new patient with slightly similar symptoms triggers that memory — but the actual cause is completely different. The vividly remembered "success story" becomes a trap that misleads her. This is the human cognitive trap: past experience distorts current judgment. The paper shows that large language models fall into the same traps.

Background: The Memory Revolution and Its Hidden Problem

Early LLMs were like goldfish — no long-term memory. By 2024–2025, memory mechanisms transformed them:

RAG: retrieving relevant information from external knowledge bases
Conversation memory: storing dialogue summaries
Knowledge-graph memory: structured fact storage
Vector databases: embedding-based semantic retrieval

Existing memory benchmarks test whether models can store, retrieve, and extract information. What they ignore is the crucial question: how does retrieved memory reshape the model's reasoning and affect performance on the current task?

Two Cognitive Traps

Trap 1: Reasoning Fixation

An AI version of functional fixedness (Duncker's candle problem): the model stubbornly applies reasoning patterns from memory even when the current task requires a different path. Example: a remembered answer about *repairing* a leaking faucet (shut the valve, replace the washer) may get recycled when the user asks about *installing* a smart faucet — even though those steps don't apply.

Trap 2: Belief Distortion

An AI version of confirmation bias: memory changes the model's factual beliefs. Example: if a dialogue history records "the solar system has nine planets," that memory can distort answers even though the model knows Pluto was reclassified in 2006. This is especially dangerous because it happens while the model "sincerely" believes it is correct, it is hard for users to detect, and it corrupts downstream reasoning chains.

MemTrapBench: Design and Findings

The benchmark's design principle is subtle: memories are faithful (correctly recorded), semantically relevant, and yet still interfere with the current task. Like giving a doctor an accurate and relevant medical record that still causes a misdiagnosis.

Experiments covered two model families (Llama and Qwen) and five memory frameworks:

| Memory framework | Mechanism | Trap susceptibility | |---|---|---| | Raw context | Direct history concatenation | High | | Summary memory | Compressed history | Medium-high | | Vector retrieval | Similarity-based snippets | Medium | | Knowledge graph | Structured facts | Medium-low | | Key-fact extraction | Only critical facts | Low (but present) |

Key finding:

> "All evaluated memory strategies underperform the no-memory baseline; even the strongest methods suffer over 10% degradation."

This is not about retrieval errors or irrelevant memories — correct and relevant memories are still harmful. A counterintuitive result: "smarter" memory is not always better; sometimes raw context beats carefully engineered summaries, because summarization itself introduces distortions.

AdaptiveMem: A Lightweight Defense

Rather than abandoning memory, the authors propose an inference-time adaptive defense:

1. Trap warning: "The following memories may contain cognitive traps that interfere with the current task" 2. Independence reminder: reason independently based on the task's specific requirements; don't over-rely on remembered reasoning patterns 3. Verification instruction: when memory conflicts with the task's requirements, prioritize the task

Results:

On MemTrapBench: significantly mitigates traps, approaching or exceeding the no-memory baseline
On standard memory benchmarks: maintains or improves performance
Works across all five memory frameworks

Deeper Implications

Why "correct" memories harm: the more correct a memory, the more the model trusts and depends on it — precisely what makes it misleading (like the veteran doctor's thirty-year-old case).
Metacognition: human memory spans working, episodic, semantic, and procedural systems, wrapped in metacognition — awareness of being potentially misled. AdaptiveMem is a first, minimal metacognitive prompt.
Lake Wobegon effect in LLMs: models either avoid indiscriminately ("I'm not sure") or answer confidently under false assumptions; few can accurately assess whether a memory applies.
Future direction: "intelligently forgetting" — memory decay, conflict detection, context tagging of when memories apply, and active forgetting of trap-forming patterns.

Takeaways

For AI builders:

More memory is not better; relevance matters more than volume
Metacognitive prompts are cheap and effective
Test not just "can it remember" but "does remembering make it worse"

For users:

Don't assume an AI's memory always helps; a "fresh" model may suit some tasks better
Be wary when an AI says "based on my experience"
Provide clear context so the model understands the task's uniqueness

Memory is a double-edged sword: a source of wisdom, or a prison of cognition. The goal should not be perfect recall but healthy skepticism about memory itself. As Borges' "Funes the Memorious" reminds us — thinking requires abstraction, and abstraction requires forgetting.

References

Wang, M., Luo, H., Xu, Z., Cui, Z., Xu, H., Yang, Q., Fang, J., Fang, J., & Zhang, N. (2026). MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use. *arXiv preprint*.

Duncker, K. (1945). On Problem Solving. *Psychological Monographs*.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Borges, J. L. (1942). Funes the Memorious. *Ficciones*.
Feynman, R. P. (1988). What Do You Care What Other People Think? W. W. Norton & Company.