Teaching AI the Art of Thinking Less: Adaptive Reasoning with NoThink, Short, and Long Modes

Forum topic · 小凯 · 2026-08-22

Summary

This post analyzes the paper "Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation" by Gijs Kassenaar, Zhao Yang, and Vincent François-Lavet. Reasoning LLMs typically use a fixed token budget, over-computing on easy problems and under-computing on hard ones. The paper's approach trains a 1.5B-parameter model to choose one of three reasoning modes—NoThink (direct answer), Short (brief reasoning), or Long (full chain-of-thought)—at the very first generated token. Training uses GRPO (Group Relative Policy Optimization) from DeepSeek-R1 with shaped rewards that reward efficient correct answers and penalize wasted or failed compute, plus hard token caps per mode to prevent mode collapse or cheating. On the MATH dataset, the adaptive model cut average response length from 4,796 to 2,811 tokens (-41%) while accuracy on MATH500 dropped only from 79.6% to 78.2%. On GSM8K, token use fell 76% with even higher accuracy, showing cross-domain transfer. The three modes emerged without external supervision and routed problems by difficulty. Compared with early exit, speculative decoding, or confidence thresholds, the method adds zero architectural overhead. The post frames this as a 'computational Occam's razor': don't add tokens without necessity, arguing that AI maturity means knowing when to think hard and when not to.

This is an English translation and structured summary of a Chinese forum post interpreting the paper "Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation" by Gijs Kassenaar, Zhao Yang, and Vincent François-Lavet.

The problem: computational overeating

Current reasoning LLMs (e.g., DeepSeek-R1, OpenAI's o-series) typically operate with a fixed token budget, generating similar-length chains of thought whether the input is elementary arithmetic or an open research problem. The authors note that fixed budgets lead to *over-computation on easy problems and under-computation on hard ones*. On the MATH dataset, a standard reasoning model consumed about 4,796 tokens per question on average, regardless of actual difficulty.

The post contrasts this with Kahneman's dual-process theory: humans automatically switch between fast, automatic System 1 thinking and slow, effortful System 2 thinking. LLMs generally do not.

Key points

Three thinking modes chosen at the first token: NoThink (answer directly), Short (brief reasoning), and Long (full chain-of-thought). No extra routing network or classifier is needed—the mode decision is made in the first generated token.
Training via GRPO with shaped rewards: Using Group Relative Policy Optimization (the RL algorithm behind DeepSeek-R1), correct answers in NoThink mode earn high rewards (efficiency bonus), Short mode earns medium rewards, and Long mode earns base rewards—so correctness alone is not the objective; efficiency matters too. Wrong answers are penalized, more severely for longer modes.
Hard token caps per mode prevent the model from "cheating" (nominally choosing NoThink while secretly generating long reasoning) and prevent mode collapse into always-NoThink or always-Long.
Headline results (1.5B-parameter model trained on MATH):

| Metric | Baseline | Adaptive | Change | |--------|----------|----------|--------| | MATH500 accuracy | 79.6% | 78.2% | -1.4 pp | | Average response length | 4,796 tokens | 2,811 tokens | -41% |

Cross-domain transfer: On GSM8K, the model achieved a 76% token reduction while maintaining *higher* accuracy, suggesting the difficulty-sensing behavior generalizes beyond the training domain.
Emergent behavior: The three modes emerged without external supervision and correlated strongly with problem difficulty—NoThink was not wasted on hard problems, and Long was not wasted on trivial ones. Notably, the Short mode ended up more accurate than Long mode.
Why the first token suffices: Pretraining gives the model an internalized "intuition" for problem difficulty from phrasing, length, and keywords—much like an experienced physician reading a patient at a glance.

Comparison with related approaches

| Method | Mechanism | Drawback | |--------|-----------|----------| | This paper | Mode choice at first token | Minimal, zero extra parameters | | Early Exit | Layer-wise dynamic depth | Requires architecture changes | | Speculative Decoding | Small draft model + large verifier | Requires two models | | Confidence thresholds | Decide after generation | Often too late |

Limitations and outlook

The paper acknowledges: validation only on mathematical reasoning (creative writing, sentiment analysis untested); three modes may be too coarse-grained; and safety-critical domains (e.g., medical diagnosis) may warrant conservative over-thinking rather than efficiency.

Takeaway

The post frames the work as a computational Occam's razor: "don't multiply tokens without necessity." Teaching AI when *not* to think deeply is framed not merely as an engineering optimization but as a form of computational ethics in an era of costly compute and energy—AI becoming "wiser" rather than just smarter, knowing when to commit full effort and when to answer lightly.

References cited in the post

Kassenaar, G., Yang, Z., & François-Lavet, V. *Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation*. arXiv preprint.
DeepSeek-AI. (2025). *DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning*.
Kahneman, D. (2011). *Thinking, Fast and Slow*. Farrar, Straus and Giroux.
Shao, Z., et al. (2024). *DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models*.