ConfSFT: Training AI Self-Awareness with 600 Problems Cuts Reasoning Tokens by Up to 25%

Forum topic · 小凯 · 2026-09-28

Summary

Researchers from the University of Maryland and Capital One propose ConfSFT (Confidence-Supervised Fine-Tuning), a self-supervised method that teaches reasoning models to predict their answer confidence mid-reasoning. Crucially, the loss function contains no length, efficiency, or stopping objective—only confidence labels. Using just 600 AIME math problems for training, the method cuts generated tokens by 10.3%–19.2% on average (up to 25%) across four model families (Gemma-4-E2B, Qwen3-4B, Nemotron-Nano-8B, GPT-OSS-20B) on math, science, and code benchmarks, with no statistically significant accuracy change. Ablations show shuffled confidence labels erase the gains, and behavioral analysis shows the method preserves the reasoning structure rather than deleting verification steps—suggesting efficient reasoning emerges as a byproduct of metacognition, without being directly optimized. Unlike inference-time early stopping (e.g., DEER), which can crash accuracy, ConfSFT requires zero changes to the inference pipeline.

ConfSFT: Teaching AI "Self-Awareness" With 600 Problems — Inference Tokens Quietly Drop by a Quarter

This post introduces a recent paper from the University of Maryland and Capital One: by training reasoning models to predict their own answer confidence mid-reasoning — with no length, efficiency, or stopping objectives in the loss function — the models become substantially more concise on their own.

Key points

The problem: Modern reasoning models (OpenAI o-series, DeepSeek-R1, Qwen3, etc.) generate long chain-of-thought drafts. RL training rewards correctness only, never penalizes verbosity, so models keep re-verifying even after reaching the answer. Example: Nemotron's median model keeps generating 1,766 tokens after confidence first exceeds 0.95.
Existing fixes have drawbacks:
*Inference-time early stopping* (DEER, etc.) can fail badly: on Nemotron, LiveCodeBench accuracy drops from 44.2% to 17.4%, HumanEval from 89.7% to 47.1%.
*Training-time length penalties* (A&Z, L1, ThinkPrune, On-Policy SFT) work but explicitly optimize for brevity and require costly RL.
The insight: A model's own confidence is an informative, self-supervised signal. On AIME2024 trajectories, higher mid-reasoning confidence correlates with higher interim-answer correctness and answer stability, while the expected benefit of continued reasoning collapses to near zero at high confidence — most of the long "tail" is statistically worthless computation.

The method: ConfSFT (Confidence-Supervised Fine-Tuning)

1. Sample reasoning traces on 600 AIME (2000–2023) problems; 8 iterations, on-policy (labels refreshed as the model changes). 2. Create confidence labels at decision points (default: before the model's natural "Wait" tokens). The tentative answer is elicited with a fixed prompt; confidence = geometric mean of its token probabilities (length-normalized), quantized to 50 percentile buckets and written as text like "64%". 3. Supervise only the label: the loss masks everything except the confidence tokens. No length terms, no stopping terms — the model only learns "given this half-finished draft, how sure am I?" 4. No inference changes: no confidence prompts, no early-stop mechanism at deployment. The conciseness comes from the weights alone.

Results

Average token reduction: Nemotron 11.1%, Gemma 10.3%, Qwen 19.2%, GPT-OSS 11.1%; up to 25% (the abstract's "up to" figure). Accuracy statistically unchanged across AIME2025, GSM8K, GPQA-Diamond, HumanEval, LiveCodeBench; pass@8 also unchanged.
Gains transfer across domains despite training only on math.
Competitive with explicit efficiency training: on Qwen, ConfSFT cuts 19.2% of tokens at 69.5 accuracy vs. On-Policy SFT's 17.2% at 69.3.
Unlike DEER, accuracy holds — and the inference pipeline stays untouched.

Ablations and analysis

Shuffled confidence labels erase or even reverse the token savings; position labels and binary correctness labels give weaker, inconsistent gains. The signal's content — calibrated-to-state, graded confidence — is what matters.
Behavioral analysis (Schoenfeld-style episodes: Read/Analyze/Plan/Implement/Explore/Verify/Monitor/Answer) shows explicit efficiency methods heavily reshape the reasoning composition, while ConfSFT's heatmap is nearly colorless: it shortens reasoning without removing any behavioral component.

Limitations

25% is a ceiling, not the average (10.3%–19.2%).
The mechanism remains a black box: why predicting confidence shortens generation lacks a causal explanation.
Confidence is self-referential and not calibrated; risk of "confidently wrong and confidently brief" on harder/out-of-distribution tasks is not fully excluded.
Largest model tested is 20B; training distribution is math-only; absolute savings match, not surpass, the best explicit methods.

Takeaway

The paper's core claim: efficient reasoning can emerge as a downstream product of learning a metacognitive signal, without being directly optimized. Teach a model to know how sure it is, and it learns on its own when it doesn't need to speak. For engineering teams this is a remarkably lightweight recipe — 600 self-supervised examples, no reward model, zero inference changes — with real cost savings at scale.

Reference: Hosseini P., et al., "Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency," https://arxiv.org/abs/2609.31619