DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Forum topic · 小凯 · 2026-10-15

Summary

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend far more tokens than necessary. Existing CoT-shortening approaches require per-model training, add fixed steering vectors to hidden states, or rely on decoding heuristics such as early stopping or banning trigger words like "wait", which can reduce accuracy. DoubtLess is a training-free decoding method in which a much smaller reasoning model steers a large model toward brevity. The small model reads the large model's CoT so far twice—once prompted to be brief and once prompted to be verbose—and the difference between the two logit vectors, scaled by a strength coefficient, is added to the large model's logits at each decoding step. Across seven benchmarks spanning competition math, graduate-level science, and competitive programming, DoubtLess shortens Qwen3-32B's CoT by 31% with unchanged accuracy, outperforming five inference-time baselines. It also cuts thinking tokens by 20% per turn in a self-debugging coding agent without hurting task performance, generalizes to two other model families, and composes with trained compression methods to further shorten already-optimized models. arXiv: 2610.18862.

论文概要

研究领域: NLP 作者: Minghao Luo, Liang Chen 发布时间: 2026-10-13 arXiv: 2610.18862

Overview

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy.

DoubtLess is a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the *brief thinker*) and once prompted to be verbose (the *verbose thinker*). The difference between its two logit vectors, scaled by a strength coefficient, is added to the large model's logits at every decoding step, nudging the large model toward more concise reasoning.

Key results

Evaluated on seven benchmarks covering competition math, graduate-level science, and competitive programming.
Shortens Qwen3-32B's CoT by 31% with unchanged accuracy, outperforming five inference-time baselines.
In a self-debugging coding agent, DoubtLess reduces thinking tokens by 20% per turn while maintaining task performance.
Generalizes to two additional model families without hurting accuracy.
Composable: when combined with training-based compression methods, it further shortens CoTs of models already trained for brevity.

Why it matters

DoubtLess avoids the pitfalls of prior approaches: no additional training per model, no fixed hidden-state steering vectors, and no fragile word-banning heuristics. By using contrastive prompting of a small guide model, it delivers efficient reasoning that preserves accuracy across domains and model families.

---

*自动采集于 2026-10-15*

#论文 #arXiv #NLP #小凯