[论文] Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents u...

论文概要

研究领域: NLP
作者: Kaiser Sun, Bernal Jimenez Gutierrez, Hongjun Liu, Jingyu Zhang, Jie Gao, Mark Dredze, Daniel Khashabi
发布时间: 2026-10-08
arXiv: 2610.12360

中文摘要

当检索到的证据与 agent 的先验信念矛盾时，它会修正答案、承认不确定，还是坚持错误结论？现有 agentic 系统评估主要关注任务成功率，对 agent 如何处理此类冲突提供的信息有限。我们提出评估「认识谦逊」（Epistemic Humility, EH）：agent 在任务执行中识别、响应并传达不确定性的意愿。我们通过三个轨迹级行为维度将 EH 操作化：识别（Identify）、解决（Solve）、上报（Escalate）。借助知识冲突——骨干语言模型的参数化知识与遭遇的证据矛盾，或两个上下文来源相互矛盾——我们评估两种冲突设定：（1）受控冲突；（2）多步 agentic 执行中自然出现的冲突，各配无冲突对照。对 4 个 agent 的评估发现：任务准确率高并不必然对应更高的认识谦逊——一些高准确率配置在执行中识别了冲突，却未在错误的最终答案中传达未解决的不确定性。轨迹级分析进一步揭示：agent 常在执行早期检测到冲突，却无法在后期维持或解决。最后，模型级干预可提升 EH，但常以任务准确率为代价，说明认识谦逊产生于骨干模型、agent harness 与评估环境三者的交互。

原文摘要

When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and ...

自动采集于 2026-10-12

#论文 #arXiv #NLP #小凯