PMR: Teaching AI What to Remember Per User via Rubric-Guided Credit Assignment
> "The cruelest thing about memory is that you cannot remember everything. Every decision to 'remember' is also a decision to 'forget.'"
This post discusses the paper *Personalized Memory Rubrics: Learning What to Remember per User via Rubric-Guided Credit Assignment* (arXiv: 2611.14017) by Yupeng Han, Kai Zhang, Shuochen Liu, et al. (USTC and City University of Hong Kong).
Key points
- The problem: Memory-operation utility varies per user. A "overwrite old preference" operation helps users with volatile preferences but harms users with stable ones. Training signals are a single terminal, binary QA reward—too coarse to credit the dozens of intermediate memory updates (add/overwrite/delete/keep) in a long conversation. This is the sparse-reward credit assignment problem, compounded by per-user differences.
- Limitations of prior work:
- Heuristic rule-based systems (Mem0, A-Mem, LightMem) use fixed, user-agnostic rules.
- RL methods (MemAgent, Mem-α, MemSkill) share one terminal reward across all updates.
- LLM-judge methods (MemCoE) score updates, but with hand-designed, scalar, identical rubrics for all users.
- Direction vs. magnitude: The terminal QA result determines credit direction; a learned rubric, applied by a frozen teacher model via token-level endorsement scores, only modulates the magnitude. A noisy rubric can thus never flip the learning direction.
- Two-layer rubrics:
- *Universal layer:* failure patterns distilled by comparing successful/failed trajectories across users.
- *Per-user residual layer:* user-specific patterns not covered by the universal layer, gated by a failure-density threshold.
- Five quality dimensions: completeness, timeliness, consistency, conciseness, relevance.
- Asymmetric design: Rubric guidance applies only to successful trajectories (credit *sorting*, robust to noise). Failed trajectories fall back to uniform GRPO (blame *attribution* is fragile to noise). Failures still update the rubric itself via Rubric Induction.
- Three gates on rubric influence:
- *Direction gate:* credit not adjusted when endorsement contradicts outcome.
- *Entropy gate:* suppresses low-surprisal "copy" tokens; credit concentrates on high-surprisal "editing" decision tokens.
- *Bounded weight:* per-token adjustments capped (e.g., ±28%).
- Co-evolution loop: rubric and student alternate—train the student under the rubric, then refresh the rubric from new trajectories. Quality control checks rubric–outcome correlation; below threshold, training reverts to standard GRPO. Frequent per-user checks (>~40% of users) are promoted to the universal layer.
- Setup: Qwen2.5-7B-Instruct backbone, trained on 8×A100 GPUs. At inference, PMR is fully self-contained—no rubric, no external evaluator, zero extra token cost.
PMR design
Results
Evaluated on four benchmarks (12 settings): PersonaMem (in-domain), PrefEval, PersonaBench, PERMA (out-of-domain). PMR ranks first in all twelve:
| Method | PersonaMem 32K | PersonaMem 128K | PrefEval explicit | PrefEval implicit | |--------|----------------|-----------------|-------------------|-------------------| | Long Context | 45.18 | 35.57 | 34.70 | 27.30 | | RAG | 54.22 | 38.50 | 47.40 | 32.40 | | Mem0 | 48.53 | 39.67 | 59.10 | 48.80 | | A-Mem | 59.63 | 40.34 | 64.70 | 54.30 | | LightMem | 54.22 | 37.40 | 67.20 | 58.90 | | MemAgent | 61.44 | 51.34 | 77.40 | 64.40 | | Mem-α | 59.63 | 49.25 | 75.70 | 63.10 | | MemSkill | 66.27 | 55.74 | 82.60 | 68.40 | | MemCoE | 67.47 | 58.67 | 83.40 | 68.60 | | PMR | 76.76 | 69.45 | 87.60 | 70.70 |
The lead over MemCoE grows from +9.29 (32K) to +10.78 (128K), supporting the hypothesis that token-level credit assignment matters more as decision chains lengthen. Ablations confirm each design choice (asymmetry, per-user layer, entropy gate, direction gate, co-evolution) contributes.
Context and takeaway
The post frames PMR as the third part of a trilogy: Cognitive LbT (limits of AI self-learning), WikiTrace (evidence chains for memory use), and PMR (credit assignment in memory learning). PMR's core insight: rubrics should be learned, but the direction of learning must never be dictated by the rubric—it is an advisor, not a judge. For large-scale personalized memory deployment, the "heavy training, light inference" property is a key engineering advantage.
Reference: Han, Y., Zhang, K., Liu, S., et al. (2026). *Personalized Memory Rubrics: Learning What to Remember per User via Rubric-Guided Credit Assignment.* arXiv:2611.14017 [cs.AI]. WSDM 2027.