Chain-of-Self-Questioning (CoSQ): A Prompt-Only Framework for Selective Abstention in LLMs

Forum topic · 小凯 · 2026-09-17

Summary

Large language models often produce fluent answers even when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit self-assessment of the information required to answer a question. The authors evaluate three CoSQ variants across seventeen conditions on the 817-item TruthfulQA multiple-choice validation set, using eleven open-weight and hosted model families. Under the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% (chain-of-thought baseline) to 8.9%, a 32.1% relative reduction, while raising answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. These improvements hold across all eleven models and every evaluated threshold. Critical-CoSQ and Adaptive-CoSQ provide adjacent operating points with 88.6% and 86.5% coverage while remaining more reliable than the baseline. Secondary experiments on Natural Questions short-answer evaluation offer converging open-form evidence. The findings show that self-questioning supports explicit, tunable answer-or-abstain decisions when unsupported commitment is costlier than deferral or review.

Paper Overview

Research area: NLP Author: Ali Şenol Release date: 2026-09-15 arXiv: 2609.17516

Abstract

Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question.

Evaluation Setup

Three CoSQ variants evaluated under seventeen conditions
Dataset: the 817-item TruthfulQA multiple-choice validation set
Models: eleven open-weight and hosted model families

Key Results

In the final balanced-option protocol:

Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% (chain-of-thought prompting) to 8.9% — a 32.1% relative reduction
Answered accuracy increases from 86.9% to 89.7%
The model still answers 87.6% of questions
Both improvements hold for all eleven models and at every evaluated threshold

Adjacent operating points:

Critical-CoSQ: 88.6% coverage
Adaptive-CoSQ: 86.5% coverage

Both variants remain more reliable than the baseline.

Additional Evidence

A secondary experiment on Natural Questions short-answer evaluation provides converging open-form evidence.

Takeaway

These findings show that self-questioning can support explicit, tunable answer-or-abstain decisions when unsupported commitment is costlier than deferral or review.

--- *Auto-collected on 2026-09-17*