[论文] When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk...

论文概要

研究领域: NLP
作者: Ali Şenol
发布时间: 2026-09-15
arXiv: 2609.17516

中文摘要

大语言模型即使在事实依据薄弱时也能生成流畅的回答。本文提出链式自我提问（Chain-of-Self-Questioning, CoSQ），一种纯提示词框架，使回答承诺以对话所需信息的明确评估为条件。我们在817题的 TruthfulQA 多选验证集上，使用11个开源和托管模型族，在17种条件下评估了三种 CoSQ 变体。在最终的平衡选项协议下，Grounded-CoSQ 在 τ=0.90 时将思维链提示下的平均无条件错误承诺率从13.1%降至8.9%（相对降低32.1%），同时将有回答的准确率从86.9%提升至89.7%，并回答了87.6%的问题。这两项改进在全部11个模型和每个评估阈值下均成立。Critical-CoSQ 和 Adaptive-CoSQ 提供了相邻的运行点，覆盖率分别为88.6%和86.5%，同时仍比基线更可靠。在 Natural Questions 短答案评估中的次要实验提供了收敛的开形式证据。这些发现表明，当不支持的承诺比转交或审查代价更高时，自我评估可以支持明确的、可调的回答或弃权决策。

原文摘要

Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. We evaluate three CoSQ variants under seventeen conditions on the 817-item TruthfulQA multiple-choice validation set using eleven open-weight and hosted model families. In the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9%, a 32.1% relative reduction, while increasing answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements hold for all eleven models and at every evalu...

自动采集于 2026-09-17

#论文 #arXiv #NLP #小凯