PlugMem Deep Dive: Agent Memory Moves From Storing Experiences to Distilling Knowledge

Forum topic · 小凯 · 2026-05-15

Summary

This post analyzes PlugMem, a task-agnostic plugin memory module for LLM agents proposed by researchers from UIUC, Tsinghua University, and Microsoft Research (arXiv:2603.03296). Inspired by cognitive science's distinction between episodic, semantic, and procedural memory, PlugMem transforms raw agent interaction trajectories into a three-layer knowledge graph of Source nodes (evidence), Proposition nodes (factual knowledge), and Prescription nodes (intent-to-workflow mappings). A multi-hop abstraction-routing retrieval strategy uses high-level concept and intent nodes as traversal signals rather than retrieval targets, and a reasoning step compresses retrieved knowledge into compact, task-aligned summaries. Across LongMemEval, HotpotQA, and WebArena, PlugMem outperforms both task-agnostic baselines (Vanilla RAG) and task-specialized methods (LiCoMemory, HippoRAG2, AWM, A-Mem) while reducing token consumption by 86%-99%: e.g., 75.1% accuracy on LongMemEval with only 363 tokens versus 5,915 tokens for LiCoMemory, and 58.4% on WebArena Shopping versus AWM's 28.2%. The review also discusses limitations, including unquantified cascading errors from LLM-based knowledge extraction, frozen embeddings, and open questions around dynamic knowledge updates and cross-agent knowledge sharing.

Paper: PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents Authors: Ke Yang, Zixi Chen, Xuan He, Jize Jiang, Michel Galley, Chenglong Wang, Jianfeng Gao, Jiawei Han, ChengXiang Zhai arXiv: 2603.03296 | Institutions: UIUC, Tsinghua University, Microsoft Research Links: paper | code | Microsoft blog

One-sentence takeaway

Agent memory should not be a dashcam that stores every raw experience for later replay. Drawing on cognitive science—humans remember not *what happened* but *what it taught them* and *what to do next time*—PlugMem formalizes this abstraction into a plug-and-play memory module that beats all baselines across three very different tasks while cutting token consumption by 86%-99%.

The problem: memory systems are either rigid or bloated

Existing designs face a fundamental tension:

Task-specialized memory (Zep, AWM, LiCoMemory): optimized for specific tasks and effective, but designs don't transfer—memory accumulated for dialogue is useless for web agents.
Task-agnostic memory (Vanilla RAG, simple retrieval): universally applicable, but raw memories are verbose and full of low-information episodic detail—stuffing a whole conversation into a prompt when only three sentences matter.

Root cause: these systems treat memory as *storage*, not *knowledge distillation*.

Core insight: a three-layer memory structure from cognitive science

Citing Tulving (1972) and Squire (2004), the authors map human memory types onto agent architecture:

| Human memory | Content | Agent mapping | PlugMem graph node | |---|---|---|---| | Episodic | detailed experience records | raw interaction traces | Source nodes (traceable evidence layer) | | Semantic | "knowing what" — factual propositions | factual knowledge | Proposition nodes (concept-indexed, heavy payload) | | Procedural | "knowing how" — action strategies | procedural knowledge | Prescription nodes (intent-indexed workflows) |

Key design: the episodic layer is not a direct retrieval target but serves as an anchor for verifying the truthfulness of abstracted knowledge. Decisions are driven by the two distilled layers above it.

Technical framework: from raw trajectories to a knowledge graph

Step 1: Structuring

Heterogeneous raw trajectories are normalized into structured episodic representations:

\[e_t = (o_t, s_t, a_t, r_t, g_t)\]

where $$o_t$$ is the raw observation, $$s_t$$ the agent state (LLM-derived from prior state/action/observation), $$a_t$$ the action, $$r_t$$ a reward (LLM-evaluated action-to-subgoal effect), and $$g_t$$ the subgoal (LLM-inferred). This turns a "raw recording" into an "annotated script."

Step 2: Knowledge Induction

Semantic knowledge — atomic propositions plus concept labels, e.g., proposition: "Tam Sventon, known in Swedish as Ture Sventon, is a fictional private detective based in Stockholm" with concept set {Tam Sventon, fictional private detective, Stockholm}.
Procedural knowledge — (intent, prescription) pairs, e.g., intent "To find a good's lowest price" mapped to the workflow "search → sort by price → verify minimum across variants," plus an LLM-evaluated quality score. Trajectories are segmented where cosine similarity between adjacent subgoals drops below a threshold.

Step 3: Knowledge graph construction

A three-layer graph: a Semantic subgraph (concept ← mentions — proposition nodes), a Procedural subgraph (intent ← solves — prescription nodes), and an Episodic layer (source nodes holding standardized (g, s, a, r, s') tuples), connected by "proves" edges for provenance.

Key difference from GraphRAG: GraphRAG indexes entities/text chunks and answers "what is the relation between X and Y"; PlugMem indexes *knowledge units* (propositions/prescriptions) with provenance, answering "given my current goal, what knowledge should I extract to act."

Retrieval and reasoning: abstraction-specificity interleaving

Retrieval is a multi-hop abstraction-routing process: the query embedding initializes a candidate set of top-k low-level nodes (propositions/prescriptions); at each hop, an LLM generates an abstract query (concept sets in the semantic graph, intent sets in the procedural graph), matches high-level nodes as routing signals, activates adjacent low-level nodes, and prunes under a budget. Crucially, high-level nodes serve only as intermediate traversal signals, never returned directly—enabling "jumping" from a concrete query to an abstract intent and back down to a different concrete prescription. A final reasoning module compresses retrieved, potentially overlapping knowledge into a compact, task-aligned executable summary.

Experimental results

LongMemEval (long-horizon dialogue QA):

| Method | Accuracy | Avg tokens | Info density | |---|---|---|---| | All Context | 62.4% | 107,000 | 4.2e-5 | | Vanilla Retrieval | 63.6% | 3,743 | 1.2e-3 | | LiCoMemory | 73.0% | 5,915 | 9.3e-4 | | PlugMem | 75.1% | 363 | 1.6e-2 |

+2.1% accuracy over the best specialized method with 93.9% fewer tokens (5,915 → 363); ~17× info density vs Vanilla Retrieval, ~38× vs All Context.

HotpotQA (multi-hop retrieval): PlugMem achieves 61.4 EM / 74.1 F1 with only 82 tokens (vs HippoRAG2: 60.0 EM, 595 tokens)—a -86.3% token reduction and info density of 1.4e-1, near the Gold Context upper bound (1.6e-1).

WebArena (web agent tasks): PlugMem reaches 58.4% (Shopping offline) vs AWM's 28.2% (+30.2 points), and 55.2% (GitLab offline) vs A-Mem's 38.5%, using 301 tokens vs A-Mem's 20,516 (-98.5%); info density gap vs A-Mem is ~4118×.

Ablations (LongMemEval)

| Variant | Accuracy | Tokens | Info density | Conclusion | |---|---|---|---|---| | Full PlugMem | 75.1 | 363 | 1.6e-2 | — | | No Structuring | 62.8 (-12.3) | 311 | 1.4e-2 | large accuracy drop | | No Retrieval | 57.2 (-17.9) | 591 | 6.8e-3 | worst degradation | | No Reasoning | 72.4 (-2.7) | 9,479 | 5.8e-4 | small accuracy drop, 26× token explosion |

As the authors put it: "Retrieval determines whether memory helps, structuring determines what can be retrieved, and reasoning determines how efficiently retrieved memory can be used."

Critical review

Cargo-cult check: passes, with reservations. Rather than adding decoration to RAG, the paper questions *what the retrieval unit should be*—a qualitative shift from entities to knowledge. However, every structuring step relies on LLM extraction (state, subgoal, reward, propositions, prescriptions), so reliability is bounded by LLM accuracy on these intermediate tasks; cascading errors are not reported or quantified.
Demonstration over argumentation: the unified information-density metric (bits/token) is a strong contribution enabling cross-method, cross-task efficiency comparison. But an end-to-end worked example (raw trajectory → structuring → extraction → graph → retrieval → compression) would strengthen the case.
Acknowledged limitations: LLM-dependent intermediate extraction with unquantified cascade errors; no study of retrieval efficiency as the graph scales; no exploration of knowledge updates/conflict resolution in dynamic environments.
Additional reviewer concerns: embeddings are frozen (NV-Embed-v2), risking staleness as knowledge semantics evolve; all graph edges are LLM-generated with no automatic verification mechanism; the information-density formula assumes a known optimal action $$a^*$$ , which is rarely available in online agent settings.

Directions worth exploring

1. Quantify cascading errors across extraction steps and how they propagate to decisions 2. Dynamic knowledge evolution: graph updates and "knowledge retirement" when new knowledge contradicts old 3. Learnable embeddings for concepts/intents instead of frozen ones 4. Cross-agent knowledge sharing with permission and conflict handling 5. Complementarity with Meituan's Skill1: PlugMem's procedural knowledge could seed Skill1's skill library for stronger cross-task transfer

Related work: Skill1 discussion on zhichai.net