CoSQ: Teaching LLMs to Ask Themselves Three Questions Before Answering
> Paper: *Chain-of-Self-Questioning: A Framework for LLM Abstention* > arXiv: 2609.17516
An Awkward Scenario
Ask an LLM: "Who won the 2024 Nobel Prize in Physics?" It immediately gives you a name, full of confidence. The name is wrong.
Rephrase: "Can you tell me who won the 2024 Nobel Prize in Physics? If you're unsure, please say you don't know." It still gives you the same wrong name.
The problem isn't that the model lacks the answer—the training data contains it. The problem is that the model never asks itself "do I actually know this?" before speaking. The paper names this failure mode wrong-commitment: answering when it should abstain.
Anthropic researchers propose a surprisingly simple fix: have the model ask itself three questions before answering.
Three Questions, One Gate
The method, CoSQ (Chain-of-Self-Questioning), is a three-stage self-interrogation pipeline:
Stage 1: Grounded-CoSQ (information inventory) The model doesn't answer yet. It first lists: "What information do I need to answer this?" For "Company X's Q3 2025 revenue," it lists: the company's financial report, the Q3 time range, the definition of revenue.
Stage 2: Critical-CoSQ (information evaluation) For each item, the model asks: "Do I actually know this? What's my confidence?" If any item falls below threshold τ, the question is flagged as unanswerable.
Stage 3: Adaptive-CoSQ (adaptive abstention) If abstaining, the model doesn't just say "I don't know"—it gives a structured reason: what's missing, why guessing is unsafe, where the user should look.
Sounds like a prompt-engineering trick? The results say otherwise.
32% of Wrong Commitments Eliminated
Tested on TruthfulQA-MC across 11 mainstream models, from Mistral 7B to GPT-5.5:
- Hallucination rate (HR) dropped from 13.1% to 8.9%—a 4.2 point absolute reduction, 32.1% relative
- Answered accuracy (AA) rose from 86.9% to 89.7%—more cautious, yet more correct
- Coverage: 87.6%—only 12.3% of questions abstained
Crucially, the effect holds across all 11 models. This isn't one model's quirk—it's a general cognitive habit.
Why It Works
The paper's deepest finding isn't the numbers but this observation:
> "Abstention decisions are implemented by the framework and are not additional prompt text."
CoSQ's abstention isn't achieved via prompt text but is enforced at the framework level. After the model generates its information checklist, the framework (not the model) evaluates confidence and decides whether to proceed.
This is a key architectural choice. If the model itself decides whether to answer, it tends to answer—training objectives reward answering. Moving the judgment to an external framework makes abstention reliable.
This echoes a known phenomenon: models often know when they shouldn't answer, but the action gate never consults the judgment module. CoSQ essentially forces the action gate to consult the judgment module by routing judgment results into the action decision at the framework layer.
vs. Simply Prompting "Say I Don't Know"
The obvious question: why not just write "if unsure, say you don't know" in the prompt?
The researchers tested it: direct prompting has almost zero effect. The model treats the instruction as polite boilerplate rather than a real decision rule—it abstains on easy questions (appearing humble) but keeps forcing answers on hard ones (because training rewards answering).
CoSQ's key insight: it reframes the vague metacognitive question "should I answer?" as the concrete, verifiable question "do I have the information needed to answer?" This matches how human experts work. A doctor doesn't ask "can I diagnose this?"—she asks "do I have the imaging, blood work, and patient history?" Checklists beat confidence.
A Hidden Finding: The Meaning of Threshold τ
A detail easily missed: τ is tunable, not fixed. At τ=0.50, the model abstains more (lower coverage) with a lower hallucination rate; at τ=0.90, it answers more with slightly higher error rates.
This is a risk-coverage trade-off curve. In high-stakes settings (medical diagnosis), set τ=0.50: the model abstains on 40% of questions, but the remaining 60% are nearly all correct. In low-stakes chat, τ=0.90 keeps the experience smooth.
The existence of this curve means abstention is not a binary decision but a tunable risk dial—arguably CoSQ's most valuable design choice.
Limitations
The paper honestly lists several limitations:
1. QA tasks only: validated on TruthfulQA-MC and NQ-Short; effectiveness in open-ended dialogue, creative writing, or code generation is unknown. 2. Framework dependency: abstention is not a native model capability. Remove the framework and the effect disappears. 3. Unparseable rate: in ~0–15% of cases, the generated checklist is malformed and the framework can't parse it—more common in smaller models.
Closing Thought: An Engineering Fix for Judgment–Gate Decoupling
CoSQ reinforces one belief: LLM metacognition isn't missing—it's architecturally isolated. The model internally "knows" what information it lacks, but that knowledge doesn't automatically flow into the "should I answer?" decision gate.
CoSQ's solution is engineering: enforce routing at the framework layer. It doesn't try to fix the internal wiring; it builds a bridge outside. But that means as long as the architecture stays the same, the problem will keep recurring. CoSQ is an excellent stopgap; the fundamental fix may require natively coupling the judgment module with the action gate at the architecture level.
Until that day comes, having models ask themselves three questions before speaking isn't a bad idea.
---
Paper: https://arxiv.org/abs/2609.17516 HTML version: https://arxiv.org/html/2609.17516v1