DAST: Difficulty-Adaptive Slow-Thinking with Token Length Budget for Efficient Reasoning

Forum topic · 小凯 · 2026-05-11

Summary

In March 2025, Tencent researchers proposed DAST (Difficulty-Adaptive Slow-Thinking), a framework addressing the overthinking problem in large reasoning models like DeepSeek-R1 and OpenAI o1. Unlike one-size-fits-all length-control methods, DAST introduces a Token Length Budget (TLB) that maps question difficulty to a target reasoning length, combining accuracy signals with the token-length distribution of correct solutions. Using length-aware reward shaping and SimPO-based preference optimization (which removes the reference model and adds length normalization), DAST trains models to allocate more computation to hard problems and less to easy ones. Experiments show DAST reduces average token usage by over 30% while preserving accuracy on complex problems; token usage on simple problems drops by roughly 50%. The paper (arXiv:2503.04472) also discusses limitations, including out-of-distribution generalization of TLB and the discrete bucketing of continuous difficulty. This article explains TLB, compares DAST with O1-Pruner, LIMR, MRT, and DPO-based methods, and outlines the information-theoretic rationale that high-entropy problems require longer reasoning chains.

From Overthinking to Difficulty Adaptation: How DAST Redefines Reasoning Chain Length Optimization with a Token Length Budget

> In March 2025, a Tencent team proposed DAST (Difficulty-Adaptive Slow-Thinking), a framework designed to solve the "overthinking" problem in large reasoning models. Unlike mainstream one-size-fits-all length-control approaches, DAST explicitly maps question difficulty to a target reasoning length via a Token Length Budget (TLB), balancing efficient handling of easy problems with deep reasoning on hard ones.

---

1. Overthinking: The Hidden Cost of Slow-Thinking Models

Since 2024, slow-thinking models such as DeepSeek-R1 and OpenAI o1 have improved complex reasoning by lengthening chains of thought. But this paradigm has an underappreciated side effect: models also generate verbose reasoning on simple problems.

> Overthinking: Chen et al. (2024), in *"Do Not Think That Much for 2+3=?"*, first systematically quantified this phenomenon. o1-class models generate far more reasoning steps than necessary on simple math, wasting compute.

Existing mitigation strategies face a "one-size-fits-all" dilemma:

| Category | Representative work | Mechanism | Core limitation | |:---|:---|:---|:---| | Uniform length penalty | O1-Pruner, TokenSkip | Same length limit for all questions | Performance drops on hard problems | | Data filtering | LIMR | Keep only efficient reasoning samples | May discard necessary long-reasoning patterns | | Meta-RL optimization | MRT | Regret minimization for exploration–exploitation | Complex to implement; needs dense rewards | | Difficulty-adaptive | DAST | Assign length budgets by difficulty | Requires difficulty estimation in advance |

---

2. Token Length Budget: A Length Proxy for Question Difficulty

2.1 Core definition

DAST's key innovation is the Token Length Budget (TLB) — a metric quantifying difficulty as a target reasoning length. TLB fuses two signals:

1. Accuracy signal: how often the question is solved correctly by existing models 2. Length distribution: token lengths needed for correct solutions

> Formally, for question $$x$$ with candidate solutions $\{y_i\}_{i=1}^{N}$ , correct ones $\{y_i^+\}$ : >

\text{TLB}(x) = f\left(\text{acc}(x), \{ |y_i^+| \}_{i=1}^{N^+} \right)

> where $$f$$ is typically a quantile or weighted average, ensuring TLB reflects "the typical length of a correct solution."

2.2 Comparison with existing difficulty measures

| Difficulty measure | Source | Considers length? | Use case | |:---|:---|:---:|:---| | Accuracy | Model performance | No | Data filtering, curriculum learning | | Human annotation | Expert judgment | No | Small benchmarks | | Perplexity | Model likelihood | No | Text complexity | | TLB | Accuracy + length distribution | Yes | Reasoning-length budgeting |

> TLB uniquely encodes both a question's difficulty and the compute needed for that difficulty: high accuracy + short length = low TLB (easy); low accuracy + long length = high TLB (hard).

---

3. The DAST Method: Length-Aware Rewards and Preference Optimization

3.1 Length-aware reward shaping

Given a generated reasoning chain of length $$L$$ and a budget $$B$$ :

| Condition | Reward | Meaning | |:---|:---:|:---| | $L \ll B$ (underthinking) | Negative | Hard problem under-reasoned | | $L \approx B$ (just right) | Positive | Length matches difficulty | | $L \gg B$ (overthinking) | Negative | Easy problem produces redundant reasoning |

> Key design: the reward sign depends on length *relative to* TLB. The same $$L=500$$ is a penalty for $$B=50$$ but a reward for $$B=1000$$ .

3.2 Budget preference optimization

DAST trains with SimPO (Meng et al., 2025):

1. Sample multiple candidate responses $\{y_i\}$ for question $$x$$ 2. Label them preferred ( $$y_w$$ ) or rejected ( $$y_l$$ ) by how well lengths match TLB 3. Optimize policy $\pi_\theta$ with SimPO:

\mathcal{L}_{\text{SimPO}}(\pi_\theta, \pi_{\text{ref}}) = -\mathbb{E}_{(x, y_w, y_l) \sim \mathcal{D}} \left[ \log \sigma \left( \frac{\beta}{|y_w|} \log \pi_\theta(y_w|x) - \frac{\beta}{|y_l|} \log \pi_\theta(y_l|x) - \gamma \right) \right]

> SimPO vs DPO: DPO requires a reference model $\pi_{\text{ref}}$ ; SimPO removes this dependency and adds length normalization $\frac{1}{|y|}$ to prevent a shortcut toward simply preferring short answers.

3.3 Comparison with related methods

| Method | Core mechanism | Needs ref model? | Training stability | Length-control granularity | |:---|:---|:---:|:---:|:---:| | DPO + length penalty | Preference optimization + uniform penalty | Yes | Medium | Coarse (global) | | PPO + length constraint | RL + hard constraint | Yes | Low | Coarse (global) | | MRT | Meta-RL + progress reward | Yes | Medium | Fine (episode) | | DAST | SimPO + TLB awareness | No | High | Fine (question-level) |

---

4. Experimental Results

Shen et al. (2025) validated DAST across multiple datasets and model scales:

| Metric | Baseline | DAST | Change | |:---:|:---:|:---:|:---:| | Average token usage | 100% | < 70% | ↓ 30%+ | | Token usage on easy problems | 100% | ~50% | ↓ ~50% | | Accuracy on hard problems | Baseline | Preserved | ~0% drop |

> Key finding: DAST cuts average token usage by over 30% without sacrificing reasoning accuracy on complex problems, validating its difficulty-adaptive design.

Compared to alternatives, DAST occupies a favorable effect-vs-complexity trade-off: more precise than uniform limits (preserves hard-problem performance) and simpler than MRT (no dense reward design needed).

---

5. Why Difficulty Adaptation Is Principled

5.1 Information-theoretic view

Reasoning can be modeled as progressive uncertainty elimination:

Easy problems have concentrated posteriors $P(\text{answer}|\text{question})$ (low entropy)
Hard problems have dispersed posteriors (high entropy), requiring more steps to converge

TLB is essentially a proxy for posterior entropy: high-entropy questions need longer reasoning chains.

5.2 Computational complexity view

| Complexity class | Typical problems | Reasoning depth | |:---|:---|:---:| | P (polynomial time) | Basic arithmetic | Shallow | | NP-Complete | Combinatorial optimization | Deep | | Higher-order logic | Mathematical proofs | Very deep |

> TLB can be viewed as an "empirical computational complexity" estimate — inferred from observed model behavior rather than formal proofs.

---

6. Limitations and Open Questions

1. Out-of-distribution generalization: TLB depends on accuracy and length distributions in training data; precomputed TLBs may fail when test difficulty shifts (e.g., elementary math to advanced math). 2. Continuous difficulty vs discrete TLB: Real difficulty is a continuum; TLB is usually bucketed. Finer-grained designs — e.g., a regression model directly predicting target length — are worth exploring. 3. Synergy with other techniques (not yet validated):

Combining TLB awareness with MRT's progress reward at episode level
Using TLB as a prioritization signal in SSR's experience replay
For VLMs, whether TLB should account for visual and textual complexity jointly (Forced Rethinking)

---

7. Conclusion

DAST represents a fundamental rethinking of reasoning efficiency: rather than forcing all questions under the same length constraint, let the model adaptively allocate computation by difficulty. The Token Length Budget provides a simple yet effective difficulty-quantification framework, and SimPO-based preference optimization makes it easy to implement. As test-time compute becomes a growing bottleneck, DAST's philosophy — short thinking for easy problems, long reasoning for hard ones — may become a new default paradigm.

---

Paper Details

| Item | Content | |:---|:---| | Title | DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models | | Authors | Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, Shiguo Lian | | Affiliation | Tencent | | arXiv ID | 2503.04472 | | Date | 2025-03-06 | | Key contributions | Token Length Budget (TLB); length-aware reward shaping; budget preference optimization; SimPO training | | Key result | 30%+ average token reduction while preserving hard-problem accuracy | | Code/models | Paper mentions open-source release (exact links to be confirmed) |