[论文] Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

论文概要

研究领域: LLM / 多智能体 作者: Anouar Imel, Imad Charradi, Abderrahman Sghir, Khalil El Khammar 发布时间: 2026-10-27 arXiv: 2610.37624

中文摘要

当LLM小组的讨论被压缩成一段简短记忆供最终决策者使用时，记忆应该保留什么？两种自然启发式方法分别保留最稀少的声明或与决策标准最相关的声明。在hidden-profile任务中，两种方法看起来都有吸引力，但事实被持有的广泛程度与其是否有用之间存在混淆。本文引入CrossProfile——一个包含60个模板、480个任务的诊断基准，其中支持性、干扰性和无关事实的归属被交叉控制。在Qwen2.5-7B、Llama-3.1-8B和两个DeepSeek模型上的实验表明：按相关性选择会重新引入干扰项（因为干扰项恰好切题），即使完美的相关性过滤器也会比rarest-first选择多损失23-31个百分点；而rarest-first选择在平均意义上不优于随机选择。两种情况下，准确率取决于记忆保留的证据平衡，而非保留策略。决策模型还引入了第三种失败——社会证明效应：给予被多数智能体归因的事实额外权重。移除归属标签或将归属随机重排可将经典配置下的准确率提升13-24个百分点。

原文摘要

When an LLM group's discussion is compressed into a short memory for a final decision maker, what should the memory keep? Two natural heuristics keep the rarest claims or the claims relevant to the decision criteria. On hidden-profile tasks both look attractive, because how widely a fact is held is confounded with whether it helps. We introduce CrossProfile, a diagnostic benchmark of 60 templates with automatically verified answer keys, in which the ownership of supporting, decoy, and irrelevant facts is crossed while their text is held fixed (480 tasks). Across Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, both heuristics fail in predictable places. Selecting by relevance readmits decoys, because decoys are on topic: even a perfect relevance filter loses 23 to 31 points to rarest-first selection in the classic hidden-profile configuration. Rarest-first selection is no better than random selection on average and wins only where the rare facts happen to be the useful ones. In both cases accuracy follows the balance of evidence the memory keeps, not the policy that kept it. The decision model adds a third failure, a form of social proof: it gives extra weight to facts attributed to many agents. Removing the attribution tags, or reassigning them at random with line order fixed, raises accuracy in the classic configuration by 13 to 24 points. In natural discussions the evidence is lost before any memory acts: the complete transcript yields lower accuracy than no notes at all.

---

*自动采集于 2026-10-29*

#论文 #arXiv #AI #小凯