[论文] Why Bother Me Again? Benchmarking Personal Agent Memory for Interactio...

论文概要

研究领域: NLP 作者: Jialin Li, Zhenhao Chen, Hanjun Luo, Hanan Salam 发布时间: 2026-10-24 arXiv: 2610.30485

中文摘要

个人智能系统正承担越来越复杂的日常任务，但反复提问、不必要的确认与不合时宜的主动联系，会把得力的助手变成交互负担。减轻负担要求智能体记住用户希望被怎样协助。然而现有记忆基准聚焦事实回忆与个性化回答，很少检验智能体如何学习并执行交互偏好。本文推出 MemPABench：评估个人智能体记忆的交互式 on-policy 环境，提出长程任务，要求智能体决定何时澄清、何时确认、何时自主行动；四位有依据的模拟用户在工作与个人场景中持有 14 项偏好，反馈横跨四个月的 99 场会话，每人五次事件驱动的偏好变化，测试记忆更新。在偏好选择、实际行为、对上下文与偏好变化的敏感性三个维度上评估四个记忆系统与四个 LLM 骨架：当前记忆带来的偏好选择增益有限——检索式系统（Mem0、MemOS、简单 RAG）不优于无记忆；文件式记忆提升准确率，但会把偏好套用到无关上下文。分析显示当前记忆难以检索交互偏好、难以适应偏好变化；即便正确偏好就在眼前，覆盖智能体的默认交互习惯依然困难。环境与数据集已开源。

原文摘要

Personal intelligence systems are taking on increasingly complex everyday tasks, yet repeated questions, unnecessary confirmations, and unwanted outreach can turn capable assistance into interaction burden. Reducing this burden requires agents to remember how users want to be assisted. However, existing memory benchmarks focus on factual recall and personalized answers and rarely test how agents learn and act on interaction preferences. We introduce MemPABench, an interactive environment for on-policy evaluation of memory in personal agents. MemPABench poses long-horizon tasks requiring agents to decide when to clarify, confirm, or act autonomously. Four grounded simulated users hold 14 preferences across work and personal contexts. Their feedback spans 99 sessions over four months, with five event-driven changes per user testing memory updates. We evaluate preference selection, delivered behavior, and sensitivity to context and preference change. Evaluating four memory systems across four LLM backbones, we find that current memory brings limited gains in preference selection. Retrieval-based systems (i.e. Mem0, MemOS, simple RAG) do not outperform No Memory, while File-based memory improves accuracy but applies preferences beyond relevant contexts. Our analysis shows that current memory struggles to retrieve interaction preferences and adapt to preference changes. Even with the correct preference available, overriding the agent's default interaction habits remains challenging. The environment and dataset are available at this https URL.

---

*自动采集于 2026-10-24*

#论文 #arXiv #NLP #小凯