PlugMem Explained: A Task-Agnostic Plugin Memory Module That Stores Experience, Not Episodes

Forum topic · 小凯 · 2026-05-15

Summary

PlugMem (arXiv:2603.03296), from UIUC, Tsinghua University, and Microsoft Research, is a task-agnostic plugin memory module for LLM agents. Inspired by cognitive science, it replaces raw episodic storage with a three-layer knowledge graph: Source nodes (episodic evidence), Proposition nodes (semantic knowledge), and Prescription nodes (procedural knowledge). Raw trajectories are standardized into (observation, state, action, reward, subgoal) tuples, then distilled into atomic propositions with concept tags and intent-prescription pairs. Retrieval uses a multi-hop abstraction-routing strategy where high-level concept and intent nodes act as intermediate signals, followed by a reasoning step that compresses retrieved knowledge into compact, task-aligned summaries. Across LongMemEval, HotpotQA, and WebArena, PlugMem beats both task-specific and task-agnostic baselines while cutting token consumption by 86%-99%. On LongMemEval it reaches 75.1% accuracy with only 363 tokens; on WebArena Shopping it hits 58.4% vs. AWM's 28.2%. Ablations show retrieval is the most critical component. The post also critiques unquantified cascading errors from LLM-dependent extraction steps and suggests directions like dynamic knowledge updates and cross-agent knowledge sharing.

PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents

Paper: PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents Authors: Ke Yang, Zixi Chen, Xuan He, Jize Jiang, Michel Galley, Chenglong Wang, Jianfeng Gao, Jiawei Han, ChengXiang Zhai arXiv: https://arxiv.org/abs/2603.03296 Institutions: UIUC, Tsinghua University, Microsoft Research

TL;DR

Agent memory should not be a dashcam that records every experience verbatim. PlugMem's insight comes from cognitive science: humans remember not "what happened that day" but "what it told me" and "what to do next time." The authors formalize this abstraction into a plug-and-play memory module that beats all baselines on three very different tasks while cutting token consumption by 86%-99%.

The Problem: Memory Systems Are Either Rigid or Bloated

Existing designs face a fundamental tension:

Task-specific memory (e.g., Zep, AWM, LiCoMemory): strong performance, but must be redesigned per task — memory accumulated in dialogue doesn't transfer to web agents.
Task-agnostic memory (e.g., vanilla RAG): general-purpose, but raw memories are verbose and full of low-information episodic detail.

The root cause: these systems treat memory as *storage* rather than *knowledge distillation*.

Core Insight: Three Layers of Memory from Cognitive Science

Drawing on Tulving (1972) and Squire (2004), PlugMem maps human memory types onto agent architecture:

| Human memory | Content | Agent mapping | PlugMem graph node | |---|---|---|---| | Episodic | detailed experience records | raw interaction trajectories | Source nodes (traceable evidence layer) | | Semantic | "knowing what" — factual propositions | factual knowledge | Proposition nodes | | Procedural | "knowing how" — action strategies | procedural knowledge | Prescription nodes |

Key design: the episodic layer is not a retrieval target; it serves as an anchor for verifying the authenticity of abstracted knowledge.

Framework: From Raw Trajectories to a Knowledge Graph

Step 1: Structuring

Heterogeneous trajectories are standardized into episodic representations:

\[e_t = (o_t, s_t, a_t, r_t, g_t)\]

where observation $$o_t$$ and action $$a_t$$ come from input, while state $$s_t$$ , reward $$r_t$$ , and subgoal $$g_t$$ are derived via LLM inference/evaluation.

Step 2: Knowledge Induction

Semantic knowledge: atomic propositions with concept tags (e.g., proposition: "Tam Sventon ... is a fictional private detective based in Stockholm"; concepts: {Tam Sventon, fictional private detective, Stockholm}).
Procedural knowledge: (intent, prescription) pairs with an LLM-scored return score. Trajectories are segmented at subgoal boundaries via cosine similarity thresholds.

Step 3: Knowledge Graph Construction

A three-layer graph: semantic subgraph (concept → proposition), procedural subgraph (intent → prescription), episodic layer (source nodes), connected by "proves" edges for provenance.

Versus GraphRAG: GraphRAG indexes entities/text chunks and answers "how are X and Y related." PlugMem indexes *knowledge units* and answers "given my current goal, what knowledge should I extract to act."

Retrieval and Reasoning: Abstraction-Specificity Interleaving

Retrieval is a multi-hop abstraction routing process: start from top-k low-level nodes (propositions/prescriptions) by embedding similarity; at each hop, generate an abstract query (concepts or intents), match high-level nodes as routing signals, activate adjacent low-level nodes, and prune to a budget. High-level nodes serve only as intermediate traversal signals. A final reasoning step distills retrieved fragments into a compact, task-aligned, executable summary.

Results

LongMemEval (long-term dialogue QA)

| Method | Accuracy | Avg. Tokens | |---|---|---| | All Context | 62.4% | 107,000 | | Vanilla Retrieval | 63.6% | 3,743 | | LiCoMemory | 73.0% | 5,915 | | PlugMem | 75.1% | 363 |

+2.1% over the best task-specific method with -93.9% tokens; ~17× information density vs. vanilla retrieval.

HotpotQA (multi-hop QA)

| Method | EM | F1 | Tokens | |---|---|---|---| | Vanilla Retrieval | 51.7 | 62.7 | 659 | | HippoRAG2 | 60.0 | 73.3 | 595 | | PlugMem | 61.4 | 74.1 | 82 |

+1.4 EM over the best specialized method with -86.3% tokens; information density (1.4e-1) near the gold-context upper bound (1.6e-1).

WebArena (web agents)

| Method | Shopping offline | GitLab offline | Tokens | |---|---|---|---| | AWM | 28.2% | 27.3% | 696 | | A-Mem | 44.3% | 38.5% | 20,516 | | PlugMem | 58.4% | 55.2% | 301 |

+30.2 points over AWM on Shopping offline; -98.5% tokens vs. A-Mem.

Ablations (LongMemEval)

| Variant | Accuracy | Tokens | |---|---|---| | Full PlugMem | 75.1 | 363 | | No Structuring | 62.8 (-12.3) | 311 | | No Retrieval | 57.2 (-17.9) | 591 | | No Reasoning | 72.4 (-2.7) | 9,479 |

The authors' summary: "Retrieval determines whether memory helps, structuring determines what can be retrieved, and reasoning determines how efficiently retrieved memory can be used."

Critical Analysis (Feynman-style)

Cargo-cult check: passed, with reservations. The paper's key move — making *knowledge* the retrieval unit rather than entities — is a genuine qualitative shift. However, all structuring steps rely on LLMs (state extraction, subgoal inference, reward evaluation, proposition/prescription extraction). The paper does not report error rates or cascading errors in these intermediate steps; an extraction error propagates into all downstream retrieval and reasoning.

Demonstration: good. The unified information-density metric (bits/token) enables cross-method, cross-task efficiency comparisons. An end-to-end worked example would strengthen the case.

Acknowledged limitations: 1. LLM-dependent extraction with unquantified cascading errors. 2. Retrieval efficiency at large graph scale unexplored. 3. Knowledge updates and conflict resolution in dynamic environments unexplored.

Additional questions raised:

The embedding model (NV-Embed-v2) is frozen — could it go stale as knowledge semantics evolve?
All graph edges are LLM-generated; is there an automatic verification mechanism?
The information-density formula assumes a perfect action label, which is rarely available in real online decision-making.

Open Directions

1. Quantifying cascading errors across extraction steps. 2. Dynamic knowledge evolution and "knowledge retirement" mechanisms. 3. Learnable embeddings for concepts and intents. 4. Cross-agent knowledge sharing over a shared graph. 5. Complementarity with Skill1 (Meituan): PlugMem's procedural knowledge could seed Skill1's skill library for stronger cross-task transfer.