Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Yields Efficient Reasoning

Forum topic · 小凯 · 2026-09-29

Summary

Reasoning models often produce overly long reasoning traces, making inference computationally expensive. This arXiv paper (2609.31619) shows that substantial efficiency gains can emerge indirectly from confidence supervision rather than explicit length optimization. Using a self-supervised procedure, the authors fine-tune reasoning models to predict their confidence in the answer at intermediate points of their own reasoning trajectories, using only 600 training problems. Confidence serves purely as a training target: the loss contains no term for reasoning length, efficiency, or stopping, and at inference the model uses standard generation without early stopping. Nevertheless, on math, science, and code benchmarks, models across the Gemma, Qwen, Nemotron, and GPT-OSS families reduce generated tokens by up to 25% while maintaining accuracy, matching gains from methods that explicitly optimize for brevity. Analysis shows confidence supervision largely preserves the base model's high-level reasoning structure rather than suppressing specific behaviors, suggesting efficient reasoning can emerge naturally as a downstream effect of learning metacognitive signals.

Paper Overview

Field: NLP Authors: Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe, Chenrui Fan, Sourya Basu, Genta Indra Winata, Anirban Das, Soheil Feizi, Nima Chitsazan arXiv: 2609.31619

Abstract (translated)

Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example via reinforcement learning with length penalties. This paper shows that substantial efficiency gains can instead emerge from a different kind of supervision: confidence.

Using a self-supervised procedure, the authors fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories, using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference time, the fine-tuned model uses standard generation, neither eliciting confidence nor applying any early-stopping mechanism.

Despite this, self-supervised confidence fine-tuning makes reasoning more efficient: on math, science, and code reasoning benchmarks, models from the Gemma, Qwen, Nemotron, and GPT-OSS families generate up to 25% fewer tokens while maintaining accuracy, matching the efficiency gains of methods that explicitly optimize for shorter reasoning.

Key Findings

Only 600 training problems are needed for the self-supervised confidence fine-tuning.
Up to 25% reduction in generated tokens at comparable accuracy across four model families.
Efficiency gains are on par with explicitly length-optimized approaches.
Analysis of reasoning traces shows confidence supervision largely preserves the base model's high-level reasoning structure rather than selectively suppressing specific behaviors.
Conclusion: efficient reasoning can emerge naturally as a downstream effect of learning metacognitive signals, without being directly optimized.

---

*Auto-collected on 2026-09-29.*