PUMA: Semantic-Preserving Early Exit Cuts Reasoning Token Waste by 26.2%

Forum topic · 小凯 · 2026-05-19

Summary

Reasoning models like o1 and DeepSeek-R1 often keep generating long chains of thought even after reaching an answer, a costly problem known as overthinking or textual inflation. The PUMA framework, introduced in arXiv paper 2605.17672, addresses this with a Semantic-Preserving Early Exit (SPEE) mechanism. A lightweight redundancy detector monitors the reasoning trajectory and, combined with an answer confidence threshold, cuts off generation when the chain of thought stops producing new information and has semantically converged. On hard benchmarks including MATH-500 and AIME, the plug-and-play monitor achieves an average 26.2% token reduction. Unlike traditional early exit based only on answer stability, PUMA preserves complete proof chains, avoiding logical gaps. Notably, forcing models to think less improved or maintained accuracy, since long CoT later stages often drift logically. The framework uses an exit decision combining redundancy detection between current and prior steps with a confidence threshold tau.

If you watch the reasoning traces of o1 or DeepSeek-R1 long enough, you get the impression of a mathematician with OCD: the answer may already be $42$ by line three, yet the model keeps circling the same semantic trap, writing a long self-justification.

This overthinking is becoming an expensive compute black hole. The PUMA framework (arXiv:2605.17672) offers an elegant solution: when reasoning converges, stop immediately.

1. Textual Inflation: Are You Paying for Filler?

There is a pathological myth in AI that longer chains of thought (CoT) mean smarter models. This "scale is justice" doctrine pushes models to run three extra miles after crossing the finish line.

> Concept note: Textual Inflation > The phenomenon where reasoning models generate far more tokens than needed to express their core logic. These redundant tokens not only increase inference cost but may also introduce logical noise.

2. PUMA: A Logical Brake for AI

PUMA introduces a Semantic-Preserving Early Exit (SPEE) mechanism, whose core weapon is a lightweight redundancy detector. It does not just watch the answer; it assesses the *quality* of the reasoning trajectory. If a detective concluded "the wall paint is gray" in the first ten steps but spends the next twenty re-describing that gray with different adjectives, the redundancy detector sounds the alarm.

Exit\_Decision = Redundant(S_t, S_{<t}) \land Confidence(A_t) \ge \tau

> Logic note: When reasoning steps no longer produce new information but instead repeat semantics, further token generation is pure waste. PUMA cuts the generation flow decisively, combined with the current answer confidence threshold $\tau$ .

3. Why This Is a Subtraction Revolution

Experiments show this plug-and-play logic monitor achieves an average 26.2% token reduction on hard benchmarks such as MATH-500 and AIME.

More ironically, forcing the model to "think less" actually improved accuracy, because the late stages of long reasoning chains often drift logically — like an accountant who computed the books correctly, then talked themselves into an error with a lengthy reflection.

4. My Bet: Refuse "Algorithmic Cargo Cult"

My bet: future top reasoning models will have built-in convergence awareness. If a model keeps rambling after convergence, it lacks true confidence in its own logic — it is merely following probabilistic inertia. If you keep paying that 26.2% redundancy cost, you are not buying intelligence, only an imitation of thinking.

The future of intelligence lies not in unlimited tokens, but in knowing when to shut up.

---

Paper Details

---

*Note: This article first appeared on zhichai.net.*