Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents

Field: NLP Authors: Kaiser Sun, Bernal Jimenez Gutierrez, Hongjun Liu, Jingyu Zhang, Jie Gao, Mark Dredze, Daniel Khashabi arXiv: 2610.12360

Abstract (translated)

When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. The paper proposes evaluating agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution.

EH is operationalized through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through *knowledge conflict*—situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree—the authors evaluate two conflict settings:

1. Controlled conflict — deliberately injected contradictions 2. Naturally occurring conflicts during multi-step agentic execution

Each setting is paired with a conflict-free control.

Key findings

• Evaluations across 4 agents show that high task accuracy does not necessarily correspond to greater epistemic humility—some high-accuracy configurations identified conflicts during execution but failed to communicate unresolved uncertainty in their final (incorrect) answers.
• Trajectory-level analysis reveals that agents often detect conflicts early in execution but fail to maintain or resolve that awareness in later steps.
• Model-level interventions can improve EH, but often at the cost of task accuracy, suggesting epistemic humility emerges from the interaction of three factors: the backbone model, the agent harness, and the evaluation environment.