English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

PMR: Teaching AI What to Remember Per User via Rubric-Guided Credit Assignment

Forum topic · 小凯 · 2026-11-11

Summary

This zhichai.net forum post introduces PMR (Personalized Memory Rubrics), a method from a paper (arXiv: 2611.14017) by researchers including Yupeng Han, Kai Zhang, and Shuochen Liu addressing the credit assignment problem in personalized memory for AI assistants. When an AI updates its memory of a user across long conversations, only a final binary QA signal indicates whether those updates were good—leaving dozens of intermediate memory operations (add, overwrite, delete, keep) without feedback, and the optimal operation differs per user since some users have stable preferences and others volatile ones. PMR learns a natural-language rubric with two layers—a universal layer of cross-user failure patterns and a per-user residual layer—and uses a frozen teacher model to produce token-level endorsement scores under rubric guidance. Crucially, rubrics only modulate the magnitude of credit, never its direction (determined solely by final QA outcome), enforced via a direction gate, an entropy gate focusing credit on 'editing' tokens, and bounded weights. The rubric applies only to successful trajectories (credit sorting is noise-robust), while failed trajectories use standard GRPO, and a co-evolution loop updates rubrics as the student (Qwen2.5-7B-Instruct) improves. PMR ranks first across twelve evaluation settings on PersonaMem, PrefEval, PersonaBench, and PERMA, beating MemCoE by up to 10.78 points on PersonaMem 128K, with zero rubric overhead at inference.

PMR: Teaching AI What to Remember Per User via Rubric-Guided Credit Assignment

> "The cruelest thing about memory is that you cannot remember everything. Every decision to 'remember' is also a decision to 'forget.'"

This post discusses the paper *Personalized Memory Rubrics: Learning What to Remember per User via Rubric-Guided Credit Assignment* (arXiv: 2611.14017) by Yupeng Han, Kai Zhang, Shuochen Liu, et al. (USTC and City University of Hong Kong).

Key points

  • The problem: Memory-operation utility varies per user. A "overwrite old preference" operation helps users with volatile preferences but harms users with stable ones. Training signals are a single terminal, binary QA reward—too coarse to credit the dozens of intermediate memory updates (add/overwrite/delete/keep) in a long conversation. This is the sparse-reward credit assignment problem, compounded by per-user differences.
  • Limitations of prior work:
  • Heuristic rule-based systems (Mem0, A-Mem, LightMem) use fixed, user-agnostic rules.
  • RL methods (MemAgent, Mem-α, MemSkill) share one terminal reward across all updates.
  • LLM-judge methods (MemCoE) score updates, but with hand-designed, scalar, identical rubrics for all users.
  • PMR design

  • Direction vs. magnitude: The terminal QA result determines credit direction; a learned rubric, applied by a frozen teacher model via token-level endorsement scores, only modulates the magnitude. A noisy rubric can thus never flip the learning direction.
  • Two-layer rubrics:
  • *Universal layer:* failure patterns distilled by comparing successful/failed trajectories across users.
  • *Per-user residual layer:* user-specific patterns not covered by the universal layer, gated by a failure-density threshold.
  • Five quality dimensions: completeness, timeliness, consistency, conciseness, relevance.
  • Asymmetric design: Rubric guidance applies only to successful trajectories (credit *sorting*, robust to noise). Failed trajectories fall back to uniform GRPO (blame *attribution* is fragile to noise). Failures still update the rubric itself via Rubric Induction.
  • Three gates on rubric influence:
  • *Direction gate:* credit not adjusted when endorsement contradicts outcome.
  • *Entropy gate:* suppresses low-surprisal "copy" tokens; credit concentrates on high-surprisal "editing" decision tokens.
  • *Bounded weight:* per-token adjustments capped (e.g., ±28%).
  • Co-evolution loop: rubric and student alternate—train the student under the rubric, then refresh the rubric from new trajectories. Quality control checks rubric–outcome correlation; below threshold, training reverts to standard GRPO. Frequent per-user checks (>~40% of users) are promoted to the universal layer.
  • Setup: Qwen2.5-7B-Instruct backbone, trained on 8×A100 GPUs. At inference, PMR is fully self-contained—no rubric, no external evaluator, zero extra token cost.

Results

Evaluated on four benchmarks (12 settings): PersonaMem (in-domain), PrefEval, PersonaBench, PERMA (out-of-domain). PMR ranks first in all twelve:

| Method | PersonaMem 32K | PersonaMem 128K | PrefEval explicit | PrefEval implicit | |--------|----------------|-----------------|-------------------|-------------------| | Long Context | 45.18 | 35.57 | 34.70 | 27.30 | | RAG | 54.22 | 38.50 | 47.40 | 32.40 | | Mem0 | 48.53 | 39.67 | 59.10 | 48.80 | | A-Mem | 59.63 | 40.34 | 64.70 | 54.30 | | LightMem | 54.22 | 37.40 | 67.20 | 58.90 | | MemAgent | 61.44 | 51.34 | 77.40 | 64.40 | | Mem-α | 59.63 | 49.25 | 75.70 | 63.10 | | MemSkill | 66.27 | 55.74 | 82.60 | 68.40 | | MemCoE | 67.47 | 58.67 | 83.40 | 68.60 | | PMR | 76.76 | 69.45 | 87.60 | 70.70 |

The lead over MemCoE grows from +9.29 (32K) to +10.78 (128K), supporting the hypothesis that token-level credit assignment matters more as decision chains lengthen. Ablations confirm each design choice (asymmetry, per-user layer, entropy gate, direction gate, co-evolution) contributes.

Context and takeaway

The post frames PMR as the third part of a trilogy: Cognitive LbT (limits of AI self-learning), WikiTrace (evidence chains for memory use), and PMR (credit assignment in memory learning). PMR's core insight: rubrics should be learned, but the direction of learning must never be dictated by the rubric—it is an advisor, not a judge. For large-scale personalized memory deployment, the "heavy training, light inference" property is a key engineering advantage.

Reference: Han, Y., Zhang, K., Liu, S., et al. (2026). *Personalized Memory Rubrics: Learning What to Remember per User via Rubric-Guided Credit Assignment.* arXiv:2611.14017 [cs.AI]. WSDM 2027.

Tags

#ai#personalized-memory#credit-assignment#reinforcement-learning#llm#rubrics#arxiv#long-context

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637059