Chain-of-Self-Questioning (CoSQ): A Prompt-Only Framework for Selective LLM Abstention

Forum topic · 小凯 · 2026-09-17

Summary

Large language models can produce fluent answers even when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. Three CoSQ variants were evaluated across seventeen conditions on the 817-item TruthfulQA multiple-choice validation set, using eleven open-weight and hosted model families. Under the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduced the mean unconditional wrong-commitment rate from 13.1% (chain-of-thought baseline) to 8.9%, a 32.1% relative reduction, while raising answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements held across all eleven models and every evaluation threshold. Critical-CoSQ and Adaptive-CoSQ offer adjacent operating points with 88.6% and 86.5% coverage while remaining more reliable than baselines. A secondary experiment on Natural Questions short answers provides convergent open-form evidence. The findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when unsupported commitment costs more than deferral or review. Paper: arXiv:2609.17516.

Paper Overview

Field: NLP Author: Ali Şenol Published: 2026-09-15 arXiv: 2609.17516

Motivation

Large language models can produce fluent answers even when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question.

Method

CoSQ is implemented purely through prompting: before committing to an answer, the model explicitly evaluates the information needed to answer the question and decides whether to answer or abstain. Three variants are studied:

Grounded-CoSQ
Critical-CoSQ
Adaptive-CoSQ

Evaluation

Dataset: 817-item TruthfulQA multiple-choice validation set
Models: eleven open-weight and hosted model families
Settings: seventeen conditions, three CoSQ variants

Key Results

Under the final balanced-option protocol, Grounded-CoSQ at τ=0.90:

Reduces the mean unconditional wrong-commitment rate from 13.1% (chain-of-thought prompting) to 8.9% — a 32.1% relative reduction
Increases answered accuracy from 86.9% to 89.7%
Answers 87.6% of questions

Both improvements hold for all eleven models and at every evaluation threshold. Critical-CoSQ and Adaptive-CoSQ provide adjacent operating points with 88.6% and 86.5% coverage respectively, while remaining more reliable than the baseline.

Additional Evidence

A secondary experiment on Natural Questions short-answer evaluation provides convergent open-form evidence.

Conclusion

These findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when unsupported commitment is more costly than deferral or review.

---

*Auto-collected on 2026-09-17*