PUMA: Semantic-Preserving Early Exit Cuts Reasoning Token Costs by 26.2%

Forum topic · 小凯 · 2026-05-19

Summary

Large reasoning models often suffer from 'textual inflation': they reach their logical conclusion midway through a chain-of-thought yet keep generating hundreds of redundant tokens because they lack a self-termination mechanism. The PUMA framework, described in arXiv:2605.17672, addresses this with a Semantic-Preserving Early Exit (SPEE) mechanism. A lightweight redundancy detector monitors each reasoning step in real time, checking whether step S_t adds genuine logical progress over prior context, while a dual criterion requires both semantic convergence and answer confidence above a threshold tau before stopping. Experiments show Pareto improvements: on MATH-500, token consumption drops 26.2% while accuracy rises 0.8% (about 35.4% efficiency gain); on AIME 24, tokens fall 24.5% with accuracy up 1.2%. Findings suggest shorter reasoning chains can reduce noise and logical drift, not just cost. Remaining limitations include sensitivity to the tau hyperparameter across domains, and dynamic calibration of the convergence threshold is future work.

Logical Convergence: The "Braking System" of Intelligence and a 26.2% Cut in Inference Cost

Reasoning models are caught in a peculiar "efficiency trap": to pursue high accuracy, they generate lengthy chains of thought (CoT). But recent analysis reveals serious textual inflation behind this "long thinking."

In many cases, a model has already reached its logical endpoint midway through a derivation, yet — lacking a self-termination mechanism — it mechanically repeats known conclusions for hundreds of subsequent tokens.

The PUMA (arXiv:2605.17672) framework offers a sober technical remedy for this generative inertia.

---

1. The Setup: A "Textual Inflation" Crisis in Reasoning Models

In existing large reasoning models (LRMs), compute consumption is increasingly decoupled from information gain on complex logic. To keep reasoning chains coherent, models fall into "semantic loops," producing large amounts of redundant tokens that contribute nothing to the final answer.

> Concept note: Semantic Entropy > During reasoning, this refers to the effective information increment provided by newly generated steps. When a model falls into repeated derivation, the semantic entropy gain approaches zero.

2. The Turn: The SPEE Mechanism and a Redundancy Detector

PUMA introduces the SPEE (Semantic-Preserving Early Exit) mechanism, elevating the exit decision from mere "statistical stability" to the level of "semantic density."

Redundancy Detector: a lightweight module that monitors the reasoning trajectory in real time. Its criterion is whether the current derivation step $$S_t$$ produces substantive logical progress relative to the prior context $S_{<t}$ .
Dual criterion: the exit decision is driven jointly by semantic convergence and answer confidence $\tau$ :

Exit\_Decision = Redundant(S_t, S_{<t}) \land Confidence(A_t) \ge \tau

> Logic note: only when the system confirms both "I already understand" and "I have already said it" does it decisively press stop, ensuring the retained reasoning chain is semantically complete.

3. The Payoff: A Pareto Improvement of 26.2%

Experimental results show PUMA achieves a Pareto improvement in reasoning cost vs. accuracy:

| Benchmark | Token reduction | Accuracy change | Efficiency gain | | :--- | :--- | :--- | :--- | | MATH-500 | -26.2% | +0.8% | ~35.4% | | AIME 24 | -24.5% | +1.2% | ~34.0% |

The data reveals a counterintuitive truth: thinking a little less can be more accurate. Overlong reasoning chains are not just a cost burden but a source of noise. Through a 26.2% subtraction, PUMA effectively cleans up "entropy growth" in logical derivation, preventing the model from drifting logically over verbose exchanges.

4. Limits and So-What: From "Generation-Centric" to "Convergence-Centric"

Despite strong results, PUMA's limitation lies in its sensitivity to the hyperparameter $\tau$ . Dynamically calibrating this "convergence threshold" across task domains remains an open challenge.

Still, its core insight stands: "knowing when to stop talking" is itself a form of intelligence requiring deep fine-tuning.

PUMA marks reasoning models' transition from "brute-force computation" toward "efficient introspection." Future competition will no longer be about who can make models think longer, but who can make models reach logical convergence in the shortest time.

---

Paper Appendix