MRAgent: Memory Is Reconstructed, Not Retrieved — Graph Memory for LLM Agents

Forum topic · 小凯 · 2026-06-16

Summary

MRAgent, a paper by Shuo Ji, Yibo Li, and Bryan Hooi (arXiv:2606.06036), argues that LLM agent memory should be actively reconstructed rather than passively retrieved, drawing on cognitive neuroscience findings that human memory is reconstructive. The system organizes memory as a heterogeneous Cue-Tag-Content graph, where lightweight Tag nodes let the LLM judge path value before reading full content, enabling early pruning. Memory access is formalized as a sequential decision process: the agent iteratively generates new retrieval cues from intermediate evidence, executes graph traversals, evaluates relevance, and decides when to stop. The paper proves theoretically that active retrieval's hypothesis class strictly contains the passive one for any budget T ≥ 2. On the LoCoMo benchmark, MRAgent (Gemini) reaches an LLM-Judge score of 84.21 versus 68.31 for Mem0, a 23% relative gain, while using only 118k prompt tokens versus 632k for A-Mem; on LongMemEval it scores 84.0 versus 65.0 for Mem0. Code is available at github.com/Ji-shuo/MRAgent.

Overview

Paper: Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents arXiv: 2606.06036 (June 2026) Authors: Shuo Ji, Yibo Li, Bryan Hooi Code: https://github.com/Ji-shuo/MRAgent

One-line summary: MRAgent makes a counterintuitive claim — existing memory systems get it wrong: memory is not looked up, it is built. Inspired by cognitive neuroscience, it turns memory access from passive one-shot retrieval into an active, multi-step, evidence-driven reconstruction process.

The Problem: Passive Retrieval

LLM agents rely on external memory systems because context windows are limited. But current methods share a blind spot: memory access is passive — a one-shot TopK selection or fixed neighborhood expansion based on the initial query, unable to adjust strategy based on evidence discovered mid-reasoning.

Example (query: "What did Nate and Caroline do in July?"):

Passive retrieval (RAG/Mem0): retrieves "Nate" → finds a gaming competition; retrieves "Caroline" → irrelevant content; fails to find Caroline's July activity.
Passive graph retrieval (A-Mem/Zep): expands neighbors from a "Nate" seed node, but Caroline has no direct edge to the competition node; still fails.
Active reconstruction (MRAgent): retrieves "Nate" → finds the competition → reasons the competition happened in July → generates a new cue "July" → retrieves July events → finds Caroline's activity. ✓

The key difference: passive retrieval only finds surface-relevant content; active reconstruction generates new retrieval cues from intermediate findings to reach indirectly relevant evidence.

Core Techniques

1. Cue-Tag-Content Heterogeneous Graph

Three node types:

Cue nodes: fine-grained keywords — names, places, times, attributes (e.g., "Nate", "July")
Tag nodes: semantic summaries of relations (e.g., "participated in", "occurred at", "likes") — letting the LLM judge path value before touching content
Content nodes: concrete memory items (e.g., "Nate entered a video game competition")

Why Tags? In a traditional knowledge graph, 1-hop expansion can explode (one name may connect to dozens of events), and each event must be read fully to judge relevance. With Tags, the agent first selects a relation ("participated in" vs "likes"), predicts content direction, prunes invalid branches, and only reads content linked via the chosen Tags — avoiding combinatorial explosion and cutting token usage drastically.

2. Multi-Layer Memory Organization

Modeled on human cognition:

| Layer | Content | Use | |---|---|---| | Episodic | Concrete events | Timeline reasoning, detail tracing | | Semantic | Abstracted knowledge | Attribute queries, preference reasoning | | Abstract | Topic summaries | Fast navigation |

3. Active Reconstruction as Sequential Decision Making

Memory access is formalized as a sequential decision process. Starting from cue nodes matched to the query, the agent iterates: (1) the LLM selects the next action (expand a Tag, retrieve content, backtrack a new cue), (2) executes the traversal, (3) evaluates relevance and prunes, (4) updates the evidence context, (5) decides whether evidence suffices.

| Dimension | Passive retrieval | Active reconstruction | |---|---|---| | Decision basis | Initial query only | Query + accumulated evidence | | Retrieval passes | One-shot | Multi-round iterative | | Cue generation | Fixed | Dynamically reasoned | | Noise control | Post-filtering | Pre-pruning |

4. Theory: Active Strictly Beats Passive

Theorem 4.1: for any retrieval budget T ≥ 2, the passive hypothesis class is strictly contained in the active one:

ℋ_passive(T) ⊊ ℋ_active(T)

Active retrieval can reach evidence spaces passive retrieval cannot.

Experiments

LoCoMo benchmark:

| Method | F1 | LLM-Judge (J) | |---|---|---| | RAG | 61.21 | 61.75 | | A-Mem | 64.45 | 68.31 | | Mem0 | 65.23 | 68.31 | | MRAgent (Gemini) | 72.26 | 84.21 | | MRAgent (Claude) | — | 76.78 |

A 23.3% relative improvement in LLM-Judge over the best baseline.

LongMemEval: MRAgent 84.0 vs Mem0 65.0 and A-Mem 62.5 (relative +32%).

Efficiency: MRAgent uses 118k prompt tokens vs 632k for A-Mem and 135k for Mem0. The key design choice: defer complex relation construction to retrieval time, done on demand.

Ablations: Full Cue-Tag-Content with reasoning reaches ~68 vs ~55 without reasoning; multi-step reasoning is the main source of gains, Tags improve retrieval precision, and the three layers are complementary. On multi-hop queries, iterative exploration boosts recall by 30%+, and the agent terminates with minimal redundant rounds.

Discussion

From library model to archaeology model: traditional systems catalog and shelve memory, then look up the catalog; MRAgent digs where current evidence suggests, using each find to decide the next dig — mirroring cognitive neuroscience's view that human memory reconstructs coherent experience from cues.
Delayed complexity: construction stays simple; relation reasoning happens at query time, where goals are explicit — more precise than blind pre-analysis at build time.
vs MemGraphRAG (KDD 2026): MemGraphRAG improves graph construction quality; MRAgent improves the retrieval process. The two are complementary.

Limitations

1. Multi-step reconstruction increases latency versus one-shot retrieval. 2. The memory graph is static after construction — no incremental updates or memory consolidation; storage grows monotonically. 3. More LLM calls per reconstruction step, which may hurt in cost-sensitive deployments. 4. The cognitive-science analogy is suggestive but LLM reasoning and neural reconstruction are mechanistically different.

Takeaway

The bottleneck of memory systems is not how sophisticated the storage structure is, but how intelligent the retrieval process is. A perfectly structured memory with rigid retrieval is like a well-stocked library with a broken catalog. MRAgent operationalizes the neuroscience concept of reconstruction so agents work like archaeologists over memory graphs — a paradigm shift, not an optimization.