DoubtLess: A Small Model as a Brake — Training-Free Elimination of Reasoning Redundancy
> Reasoning models (like DeepSeek-R1, Qwen3) have a notorious problem: overthinking. For a high school math problem, they can write 3000 tokens of "let me think...". A team from CUHK and EPFL proposed a clever method: use a small model as a "brake," stepping in when the large model starts rambling. No training required, and accuracy doesn't drop.
A Daily Scenario: Two People Discussing a Problem
Imagine you have a very smart friend, but he has a flaw—his mouth won't stop when thinking. Ask him "what is 7 times 8," and he won't just say 56. Instead: "Let me think, 7 times 8, we can decompose 7 into... no, actually a simpler way is... wait, let me verify..."
You also have a friend who's less brilliant but very direct. Ask the same question, and he says "56."
DoubtLess's idea: let the direct friend sit nearby, and gently pull back the smart friend when he starts going in circles.
The "Overthinking" Disease of Reasoning Models
Reasoning models solve problems through long chains of thought (Chain-of-Thought, CoT). This is powerful, but has a side effect: the model often keeps generating large amounts of "verification" and "repeated confirmation" tokens after it has already found the answer.
Existing solutions all have problems:
| Method | Problem | |--------|---------| | Training-based compression | Requires separate training per model, high cost | | Fixed steering vectors | Applies the same compression force to all questions, inflexible | | Banning words like "wait" | Too crude; sometimes "wait" is necessary self-correction | | Early stopping | May miss crucial verification steps |
The common problem of these methods is that they are one-size-fits-all—they don't distinguish "necessary thinking" from "redundant hesitation."
DoubtLess's Core Design: Two "Thinkers"
DoubtLess's method is extremely simple. It uses a small model (e.g., a 0.5B or 1.5B reasoning model) to guide a large model (e.g., Qwen3-32B). The steps:
Step 1: The small model reads the CoT the large model has generated so far.
Step 2: The small model does two inference passes with this CoT:
- Brief thinker: prompt is "please answer briefly"
- Verbose thinker: prompt is "please answer in detail"
- Qwen3-32B's CoT shortened by 31%, with unchanged accuracy
- Outperforms 5 inference-time compression baselines (NoWait, DEER, ThinkBrake, ASC, etc.)
- Compression remains stable on harder problems
- Also effective in coding agent scenarios
Step 3: Compute the difference between the two thinkers' logit vectors, multiplied by a strength coefficient.
Step 4: Add this difference to the large model's logits.
That's it. No training, no modification of the large model's weights, no extra data.
Why This Works: The Contrastive Decoding Intuition
The key intuition: the "disagreement" between the two thinkers points toward the "redundancy direction".
When the large model is already close to the answer, the brief thinker gives the answer directly, while the verbose thinker keeps elaborating. Their logit difference points toward the "keep elaborating" direction—exactly the direction in which the large model is hesitating.
Adding this difference in reverse to the large model says: "Hey, that small model thinks this can be more concise—consider it."
This aligns with the principle of contrastive decoding: use a "worse" model's output as a counterexample to steer the main model away from certain behaviors. But DoubtLess's innovation is that it doesn't use a worse model as a counterexample; it uses the difference between two versions of the same model under different prompts as the directional signal.
Experimental Results: 31% Compression, Accuracy Unchanged
Tested on 7 benchmarks (competition math, graduate-level science, competitive programming):
The Most Interesting Finding: The Deleted Tokens Are All After the Answer
The analysis in Section 5.1 is particularly insightful. They traced exactly which tokens DoubtLess deletes and found:
Most deleted tokens appear after the answer first appears.
This shows that reasoning models' "overthinking" isn't in the "finding the answer" phase, but in the "already found it but unsure whether to trust it" phase. Tokens like "let me verify" and "wait, I need to confirm" generated after the answer—these are the redundant ones.
Even more interesting: the two thinkers almost agree before the answer appears, but start to diverge after it. The brief thinker says "stop here"; the verbose thinker says "keep going." This divergence is the signal DoubtLess uses to guide the large model.
The paper also gives an intuitive explanation: "hesitant words are kept". DoubtLess doesn't delete functional words like "wait" (which are sometimes necessary self-correction signals), but rather the redundant verification after the answer. This is much smarter than simply banning "wait."
Complementarity with Training-Based Compression
Section 5.2 of the paper does an important experiment: DoubtLess can be stacked on top of training-based compression methods.
For example, a model already trained to reason more briefly can be compressed further with DoubtLess. This shows the two act on "redundancy at different levels"—training-based compression eliminates "model-level redundant tendencies," while DoubtLess eliminates "per-question redundant tendencies."
Technical Details: Three Key Design Choices
1. Where the contrast applies
DoubtLess doesn't apply contrast to all tokens. The paper discusses the choice of scope—applying it only to the CoT part, without affecting final answer generation.
2. When it acts
Not from the very beginning. The small model needs to read enough context to make meaningful judgments. The paper discusses the choice of activation timing.
3. Strength selection
A strength coefficient too small has no effect; too large harms accuracy. Appendix F of the paper gives a methodology for strength selection and the impact of small model size.
Code Is Open Source
Unlike many papers, DoubtLess's code is open source: https://github.com/leoluolol/DoubtLess
This means you can try it directly on your own reasoning models.
Why This Matters
Inference cost is one of the biggest bottlenecks in current AI deployment. A reasoning model's cost to solve a problem can be 10x that of a non-reasoning model, mainly because the CoT is so long.
DoubtLess's value: 1. Zero training cost: no additional training, plug and play 2. Zero accuracy loss: accuracy unchanged on all benchmarks 3. Stackable: complementary to training-based compression methods 4. Interpretable: the deleted tokens follow a clear pattern (redundant verification after the answer); not a black box
From a broader perspective, DoubtLess represents a class of approaches: using small models to guide large models. This isn't new (contrastive decoding etc. have similar ideas), but DoubtLess's "same model, two prompt versions" design is one of the simplest.
An Honest Assessment
A few limitations:
1. Requires a small model: though cheap (0.5B–1.5B), it's still extra inference overhead. The paper discusses inference cost—the small model's forward pass adds per-step overhead, but overall net cost still drops because CoT shrinks by 31% 2. Choice of "brief" and "verbose" prompts: the paper uses simple prompts, but whether more sophisticated prompts could give better guidance wasn't explored 3. Only tested on math/science/coding: effect on open-ended generation (writing, dialogue) is unknown 4. 31% compression rate: significant, but some training-based methods reach 50%+. DoubtLess's advantage is zero training cost
Overall, DoubtLess is a "small but beautiful" piece of work—simple method, clear results, open-source code, thorough analysis. It won't overturn the reasoning model training paradigm, but it provides a practical, plug-and-play inference optimization tool.
---
Paper: DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning Authors: Minghao Luo (CUHK), Liang Chen (EPFL) Code: github.com/leoluolol/DoubtLess