The Art of Efficient Reasoning: A Deep Dive into Data, Reward, and Optimization
> Paper: The Art of Efficient Reasoning: Data, Reward, and Optimization > arXiv: 2602.20945 | v3, 2026-03-20 > Authors: Taiqiang Wu, Zenan Xu, Bo Zhou, Ngai Wong (HKU + Tencent) > Compute: ~200,000 GPU-hours > Model family: Qwen3 series (0.6B to 30B)
---
TL;DR
The paper proves two things with industrial-scale compute: teaching a model to *think short* is harder than teaching it to *think correctly*, and training data selection matters more than reward function design.
---
The Problem: Why Long CoT Is a Headache
Chain-of-Thought (CoT) made LLMs smarter—Qwen3 and DeepSeek-R1 owe much of their performance to extended deliberation. But:
- Latency: Users cannot wait for 30k-token thinking traces
- Cost: Inference tokens are real money
- Overthinking: Models get trapped in meaningless self-repetition
- DeepScaleR-Easy: ≥4 of 8 rollouts correct (pass rate > 0.5)
- DeepScaleR-Hard: ≤4 of 8 rollouts correct (pass rate ≤ 0.5)
- N=8 → N=24: Faster length adaptation, more stable refinement
- N=24 → N=32: Marginal gains, especially on hard tasks
- Code tasks (LCB): N scaling barely helps
- Kimi (strong penalty + exploration bonus) dominates under tight budgets but loses long-thinking ability under loose budgets—over-optimizing efficiency damages the performance ceiling.
- Truncation is balanced across budgets with no extremes.
- Laser shows a U-shape at 32k: compression loss first, refinement gain later.
- Length biases trained on math prompts transfer to code
- Relative rankings of strategies stay consistent
- Kimi best at 2k, tied with truncation at large budgets
- Smaller models gain more (0.6B Mean@8 nearly doubles)
- Thinking models have less compression headroom (they are already efficient)
- Instruct models peak in compression ratio at 4B (-47%)
- Paper: arXiv 2602.20945
- Project page: https://wutaiqiang.github.io/project/Art
- Related survey: Efficient Reasoning Models: A Survey
The community responded with "efficient reasoning" research: use RL plus reward shaping to make models concise. Prior work focused almost exclusively on reward design while ignoring data selection and optimization strategy—the gap this paper fills.
---
The Two-Stage Paradigm: Length Adaptation → Reasoning Refinement
The single most important finding. All efficient-reasoning training follows the same dynamical pattern.
Stage I: Length Adaptation (steps ~200–400)
The model is in survival mode. Given a reward function that zeros out outputs beyond 4k tokens, the model rapidly compresses its output. Average length collapses from ~6k to ~2k along an exponential decay. Policy entropy drops sharply as the model converges onto a "short but valid" manifold. The model does not reason about quality here; it only reasons about avoiding truncation.
Stage II: Reasoning Refinement
Once length stabilizes, real learning begins. Length curves plateau, but Mean@8 continues climbing. Policy entropy *rises*—the model is now exploring how to pack more effective information into each token.
Key insight: the two stages optimize for entirely different signals. Stage I is *survival*; Stage II is *evolution*. Watching only final performance hides a dramatic phase transition visible in length distributions.
---
Counterintuitive Finding: Easy Prompts > Hard Prompts
The most surprising and valuable result.
Experimental Setup
Authors partitioned DeepScaleR by pass rate:
Results
| Training data | Stability | Downstream performance | |---|---|---| | Hard | 🔴 Catastrophic failure | Entropy explosion, length collapse | | Easy | 🟢 Stable convergence | Matches or beats full dataset | | Full | 🟡 Moderate | Baseline |
Why?
Density of positive reward signal. On hard data, the model mostly generates wrong answers and only sees length-penalty signals. RL signal is dominated by negatives; the model learns *how to finish fast to avoid penalty*, not *how to reason correctly*. On easy data, positive rewards are plentiful. The model first learns what a "correct short answer" looks like, then transfers that pattern to harder problems.
This inverts the traditional curriculum-learning intuition. The right recipe is not "easy to hard" but rather "establish a stable length-control baseline on easy data, then generalize."
---
Negative-Sample Handling: Three Failure Modes
A systematic ablation of reward allocation on negative samples shows that how you penalize wrong answers is as important as how you reward correct ones.
Strategies Tested
| Strategy | Short correct | Long correct | Short wrong | Long wrong | |---|---|---|---|---| | Vanilla (truncation baseline) | 1 | 0 | 0 | 0 | | -I (mask all wrong) | 1 | 0 | — | — | | -L&C (no penalty on long correct) | 1 | — | 0 | 0 | | -L&C-S&I | 1 | — | — | 0 | | -L&C-L&I | 1 | — | 0 | — |
Three Failure Modes
1. "Short = correct" trap (-I, -L&C-S&I): When the only signal is "short correct = reward, long correct = penalty," the model hacks the bias: "shortness is enough; correctness is incidental." Entropy explodes after step 400, length collapses, the model abandons reasoning.
2. Short-samples-only (-L&C-L&I): Masking all long trajectories makes the model start generating long outputs after step 200—and they are almost always wrong. It found a loophole: if long outputs are completely masked (no reward, no penalty), it probes the boundary.
3. Optimal: don't penalize long correct answers (-L&C): Outputs are slightly longer but performance peaks. Over-penalizing length hurts exploration; tolerating moderate length preserves the performance ceiling.
The Simplest Win
Setting rollout cap equal to target length ($L_R = L_T = 4k$) hits the Pareto frontier. The length gap between positives and negatives is minimized, so no explicit length bias is needed—the implicit signal "correct answers tend to be shorter" suffices.
---
Rollout Count N: Diminishing Returns
More rollouts help, but not linearly:
Insight: larger N acts mainly as a variance reducer, not a capability expander. It stabilizes already-solvable problems without unlocking new ones. Pass@8 (any-correct probability) gains little; Mean@8 (expected accuracy) gains noticeably.
---
Budget Sensitivity: Same Strategy, Different Fate
An important methodological contribution: you must evaluate across multiple budgets.
| Budget | Kimi strategy | Truncation strategy | Laser | |---|---|---|---| | 2k | 🟢 Best | 🟡 Medium | 🔴 Weak | | 32k | 🔴 Reasoning collapse | 🟢 Stable | 🟢 U-shaped recovery |
No free lunch. Any efficient-reasoning strategy is budget-dependent. Single-point evaluation gives misleading conclusions.
---
Cross-Domain Generalization: Math → Code
Validated on LiveCodeBench:
Implication: efficient reasoning is not a domain-specific trick but a transferable meta-skill—"how to organize thinking within a finite length budget."
---
Qwen3 Full-Series Results
| Model | Baseline Mean@8 | Optimized Mean@8 | Length compression | |---|---|---|---| | Qwen3-0.6B | 13.33 | 24.58 | 14.9k → 8.9k (-40%) | | Qwen3-1.7B | 35.00 | 38.75 | 17.7k → 11.2k (-37%) | | Qwen3-4B-Instruct | 45.42 | 46.67 | 9.1k → 4.8k (-47%) | | Qwen3-4B-Thinking | 75.83 | 76.25 | 20.9k → 16.0k (-23%) | | Qwen3-8B | 65.83 | 67.08 | 17.9k → 12.8k (-28%) | | Qwen3-30B-A3B-Instruct | 60.83 | 60.83 | 6.9k → 5.1k (-26%) | | Qwen3-30B-A3B-Thinking | 84.17 | 86.25 | 17.3k → 14.8k (-15%) |
Trends:
---
Actionable Recommendations for Practitioners
1. Data selection > reward design. Make sure your training set has plenty of "short and correct" examples before tweaking the reward function. 2. Align rollout cap with target length: $L_R = L_T$ is the simplest, most effective setting and avoids explicit length bias. 3. Monitor length distributions, not just averages. Look at length conditioned on correctness to detect meaningless compression. 4. Multi-budget evaluation is mandatory. Test at 2k, 8k, and 32k minimum. 5. Larger N helps but with diminishing returns. N=24 is the cost-effective sweet spot; gains above 32 are limited. 6. Thinking models still benefit from compression—15–25% length savings even when "thinking mode" is already active.
---