[论文] Ensemble Methodology for Reference-Free Confidence Estimation in LLM-b...

论文概要

研究领域: ML 作者: Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, Animesh Kumar 发布时间: 2026-11-06 arXiv: 2611.09846

中文摘要

本文描述了一种在无法获取人类撰写参考的情况下估计 LLM 生成的 COBOL 摘要正确性的方法和用例。核心直觉是：模型间在语义上的一致程度能反映真实性，LLM-as-judge 能提供比传统字符串或嵌入指标更高保真度的成对评估。作者将这一直觉操作化为：(i) 从多样化模型受控生成摘要；(ii) LLM 评估器比较摘要对并输出结构化判断；(iii) 校准的聚合规则结合支持性证据，同时只惩罚经证实的矛盾。

原文摘要

This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. We operationalise this intuition with (i) controlled summary generation from diverse models; (ii) a LLM evaluator that compares pairs of summaries and emits structured judgments; and (iii) a calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.

#论文 #arXiv #ML #小凯