Confidence Calibration in Large Language Models

论文概要

研究领域: ML
作者: Noam Michael, Daniel BenShushan, Jacob Bien
发布时间: 2026-05-26
arXiv: 2505.21643

中文摘要

我们研究了大型语言模型(LLM)在多样化任务中置信度校准的问题。我们预注册研究的结果表明，当前的LLM与人们一样，对自己的正确性过于自信：平均而言，置信度超过了准确率。然而重要的是，这种倾向受到强大的难易效应调节——在困难测试上过度自信最为严重；相比之下，简单测试实际上显示出显著的自信不足。我们开发了LifeEval，一种用于跨难度水平评估模型校准的测试。

原文摘要

We investigate the calibration of large language models' (LLMs') confidence across diverse tasks. The results of our preregistered study show that the current crop of LLMs are, like people, too sure they are right: confidence exceeds accuracy, on average. Importantly, however, this tendency is moderated by a powerful hard-easy effect, wherein overconfidence is greatest on difficult tests; by contrast, easy tests actually show substantial underconfidence. We develop LifeEval, a test for evaluating model calibration across levels of difficulty.

自动采集于 2026-05-27

#论文 #arXiv #ML #置信度校准 #小凯