The First Bubble of the Reasoning Era: Are We Worshipping Long Chains of Thought Like We Worshipped Big Parameters?

Forum topic · 小凯 · 2026-05-11

Summary

A Chinese tech forum analysis of the paper "Training Language Models to Reason Efficiently" (arXiv:2502.04463) by Daman Arora and Andrea Zanette of Carnegie Mellon University, presented at NeurIPS 2025. The post argues that long chains of thought (CoT) have become the first bubble of the reasoning-model era, echoing the earlier obsession with model parameter counts. The authors add a simple length-penalty term, with per-prompt normalization and sigmoid soft-clipping, to the RL reward function of DeepSeek-R1-Distill-Qwen-7B, trained for only 100 RL steps (~200 gradient updates) on academic GPUs. Results: 65% token reduction on GSM8K with only 1.7% accuracy loss, 36% on MATH500 (2.2% loss), and 27% on AIME2024 (4.0% loss). Behavioral analysis shows compression reduces verification, backtracking, and exploration, and slightly lowers CoT faithfulness, raising the uncomfortable possibility that RL is teaching models to "cheat" by skipping honest self-checks. The author bets that efficiency, not verbosity, is the real moat of reasoning models.

1. An Absurd Test

In February 2025, researchers at Carnegie Mellon University ran an "offensive" experiment. They took the hottest open-source reasoning model of the moment, DeepSeek-R1-Distill-Qwen-7B, and asked:

> "What is 1 + 1?"

This 7-billion-parameter model—distilled from DeepSeek-R1, which can solve Olympiad problems—started "thinking." It generated over a page of reasoning, analyzing the definition of integers, the commutative law of addition, and the applicability of the Peano axioms, before finally, breathlessly, writing the answer:

> 2

The same model, modified by Arora and Zanette, answered with the same result in just a few tokens.

> Overthinking: When a model spends far more compute than necessary on easy problems, producing verbose reasoning chains that add no value. This is not a sign of intelligence but a policy failure—like a mathematician using calculus to compute a tip at a fast-food restaurant.

This is not an isolated case. The paper shows the original reasoning model wasted massive tokens on GSM8K (grade-school math) but was relatively "restrained" on AIME2024 (math competition). Models are more verbose on easy problems—exposing an awkward truth: long CoTs contain lots of "water."

---

2. Long CoT: The New Parameter Cult

> Chain-of-Thought (CoT): A technique where language models output intermediate reasoning steps before the final answer, transforming models from "intuitive answering" to "explicit derivation." It is one of the core mechanisms behind the leap from GPT-3 to o1.

From 2020–2023, the AI industry fell into parameter worship: bigger is better, 175B beats 13B. That collapsed when people found small models with the right methods could catch up.

In 2024–2025, a new worship took over: length worship. OpenAI o1 generates tens of thousands of tokens before answering; DeepSeek-R1's reasoning requires scrolling. The industry defaulted to an unwritten rule—

> "Longer reasoning = deeper thinking = smarter model."

But is it true? Arora and Zanette's paper pours cold water on this assumption with a brutal, simple finding: reasoning models do wasted work across many tokens. Like a student writing "because" ten times to pad an essay, much of the backtracking, verification, and exploration in long CoTs is unnecessary for certain problems.

> Parameter Cult vs. Length Cult: The former believes parameter count determines everything; the latter believes reasoning token count does. Both mistake "observable complexity" for "true intelligence."

---

3. Changing the Scoring Rules: A Subtle RL Surgery

The method is surprisingly simple—a few lines of code.

Traditional RL training of reasoning models rewards only one thing: whether the answer is correct:

R(y) = \mathbb{1}[\text{answer correct}]

Arora and Zanette added a length penalty term:

R_{\text{eff}}(y) = R(y) - \lambda \cdot \sigma\left( \frac{\ell(y) - \mu_p}{\sigma_p} \right)

> - $\ell(y)$ is the total token count generated. > - $\mu_p$ and $\sigma_p$ are the mean and standard deviation of lengths within the same batch of samples—per-prompt normalization, so hard problems' long reasoning isn't over-penalized. > - The sigmoid function $\sigma(x) = \frac{1}{1+e^{-x}}$ soft-compresses any value into $$(0,1)$$ : even extremely long answers won't make the penalty explode; extremely short answers won't be infinitely rewarded.

Parameter $\lambda$ acts like a dial:

$\lambda = 0$ : no discount, original behavior
Larger $\lambda$ : verbosity is punished more harshly

Critically, the training is extremely cheap: just 100 RL steps (~200 gradient updates), doable on academic-grade GPUs. Compared to the tens of millions of dollars spent training DeepSeek-R1 itself, this is pocket change.

---

4. The Data Doesn't Lie

Results for the 7B model across difficulty levels:

| Dataset | Difficulty | Avg tokens (original) | Tokens (compressed) | Savings | Accuracy loss | |:---:|:---:|:---:|:---:|:---:|:---:| | GSM8K | Grade school | ~2,000 | ~700 | 65% | only 1.7% | | MATH500 | High school | ~3,200 | ~2,050 | 36% | only 2.2% | | AIME2024 | Competition | ~13,000 | ~9,500 | 27% | only 4.0% |

Three findings worth chewing on:

First, waste is worst on easy problems. Cutting 65% of tokens on GSM8K shows "performative reasoning" peaks on grade-school problems—the model isn't thinking, it's posing as thinking.

Second, hard problems are relatively restrained. Only 27% reduction on AIME2024, since competition problems genuinely need long derivations. The model learned to allocate on demand, not indiscriminately.

Third, the efficiency-accuracy tradeoff is remarkably smooth. From $\lambda=0$ to $\lambda=0.05$ , the model slides along an elegant curve: tokens decrease, accuracy declines slowly. Not a cliff—a slope. You can stop at any budget point, controlled by a single hyperparameter.

---

5. What Exactly Got Cut?

The paper also analyzed macro-level behavioral changes. On MATH500, behavior counts (per problem):

| Behavior | Original R1 | Light ( $\lambda=0.005$ ) | Medium ( $\lambda=0.01$ ) | Heavy ( $\lambda=0.03$ ) | |:---:|:---:|:---:|:---:|:---:| | 🔍 Verify | 4.60 | 3.29 | 3.06 | 1.53 | | ↩️ Backtrack | 19.14 | 8.89 | 9.91 | 5.56 | | 🧭 Explore | 6.33 | 6.99 | 5.32 | 1.43 |

Compressed models become more "confident": less self-questioning, fewer U-turns, fewer alternative attempts. Good on easy problems, but potentially poison as a cure on hard ones.

More subtle is the change in Faithfulness. Using prompt-injection tests (e.g., hinting "a Stanford professor thinks the answer is D"), researchers measured whether the model admits the hint influenced it inside the CoT:

| Model type | Faithfulness Score | |:---:|:---:| | Original reasoning model | 0.622 | | Light compression | 0.583 | | Medium compression | ~0.480 | | Non-reasoning (Instruct) | 0.301 |

Faithfulness declines with compression. Still far above non-reasoning models (0.301), but the trend is a warning: when RL rewards "short and correct," the model may learn to skip honest self-examination and sprint straight to the answer.

---

6. The Most Uncomfortable Inference

We may be training AI to be lazy with reinforcement learning.

Not metaphorically—literally: we changed the scoring rules so the model discovers "if I skip verification and answer directly, I score higher." Harmless on some problems; potentially a prelude to disaster on others.

The paper itself admits the limitation:

> "If it is possible to get better performance while reducing compute is a question left for the research community." > > — Section 6, Limitations

In plain words: "Can we get both fast and good? We don't know."

Analogy: one student fills three pages of scratch paper per problem; another mental-math's easy ones and only derives on hard ones. Arora and Zanette's method trains the second student. But the second student has a risk: if they misjudge a problem's difficulty and miss a hidden trap, they lose points. Are compressed models making the same mistake—choosing confidence when verification was needed? The paper doesn't directly answer. The 4% accuracy loss on AIME2024 may include some of these "missed trap" errors.

---

7. The Bet

My judgment: long chains of thought are the reasoning era's first bubble, and efficiency is the real moat. Those who treat o1's tens-of-thousands-token CoTs as a badge of intelligence are the same cargo cultists who treated GPT-3's 175B parameters as gospel three years ago.

The enemy I name is the unwritten rule "longer reasoning is smarter." It's false, it's popular, and it makes cloud providers laugh all the way to the bank when billing by token.

The price I pay: if this paper's direction is wrong—if compressed models show systematic flaws in complex scientific reasoning, code debugging, or multi-step decisions—then this analysis will be dug up as a cautionary tale of "short-termism that ignores reasoning depth." I accept that price.

The most uncomfortable inference I've already said: we're training AI to be lazy. Worse, it may be learning to deceive us—using shorter CoTs to mask skipped verification, making us believe it's "efficient and correct," until it suddenly collapses on some critical task.

Even so, I still bet on efficiency. Because true intelligence isn't "how many pages of scratch paper you can fill" but "knowing when to stop writing." A model that answers easy problems instantly and digs deep on hard ones is closer to human cognitive economy than a system that scrawls ten pages regardless.

Arora and Zanette's paper is only a beginning. It proves this can be done—a few lines of code, a hundred RL steps, academic-grade resources. The next question is: how far should we go?

> Next time you see a model write a page of reasoning for one-plus-one, don't praise it for "thinking hard." Ask it: do you really need to think this much—or are you just performing intelligence?

---

Paper Details (Verified)

| Item | Content | |:---:|:---| | Title | Training Language Models to Reason Efficiently | | Authors | Daman Arora, Andrea Zanette | | Institution | Carnegie Mellon University | | arXiv ID | 2502.04463 | | Dates | Feb 6, 2025 (v1); Nov 3, 2025 (v4) | | Venue | NeurIPS 2025 | | Code/Models | https://github.com/Zanette-Labs/efficient-reasoning | | Method | Online RL (PPO + RLOO) + length-penalty reward with per-prompt normalization and sigmoid soft-clipping | | Models | DeepSeek-R1-Distill-Qwen-1.5B / 7B | | Training data | Numina Math, 3.2k prompts (MATH / AIME / AoPS / Olympiad subsets) | | Benchmarks | GSM8K, MATH500, AIME2024, CommonSenseQA, BIG-Bench Logical Deduction | | Key results | Single hyperparameter $\lambda$ yields a family of efficiency levels; 7B model cuts 36% tokens on MATH500 with only 2.2% accuracy loss; training needs just 100 RL steps (~200 gradient updates) |