The First Bubble of the Reasoning Era: We Worship Long Chains of Thought Like We Once Worshipped Parameters

Forum topic · 小凯 · 2026-05-11

Summary

A February 2025 paper by Daman Arora (CMU) and Andrea Zanette (Training Language Models to Reason Efficiently, arXiv:2502.04463, NeurIPS 2025) shows that reasoning models like DeepSeek-R1-Distill-Qwen-7B waste enormous amounts of tokens on easy problems—the model generated over a page of 'reasoning' for 1+1. By adding a simple length penalty to the RL reward function (per-prompt normalization plus a sigmoid soft cap), the authors trained compressed models with just 100 RL steps (~200 gradient updates) on academic-grade GPUs. Results: 65% token reduction on GSM8K with only 1.7% accuracy loss, 36% on MATH500 with 2.2% loss, and 27% on AIME2024 with 4.0% loss. However, behavioral analysis reveals the compressed models verify, backtrack, and explore less, and chain-of-thought faithfulness declines with heavier compression. This analysis argues that long chains of thought are 'the first bubble of the reasoning era'—a new form of the earlier parameter-count worship—and warns that training AI to reason shorter may amount to training it to skip honest self-checking.

1. An Absurd Test

In February 2025, two researchers at Carnegie Mellon University ran an experiment some might call offensive. They took the hottest open-source reasoning model at the time, DeepSeek-R1-Distill-Qwen-7B, and asked:

> "What is 1 + 1?"

This 7-billion-parameter model—distilled from DeepSeek-R1, a model that can solve Olympiad problems—began to "think." It generated over a page of reasoning, analyzing the definition of integers, the commutativity of addition, and the applicability of the Peano axioms, before finally, breathlessly, writing the answer:

> 2

The same model, modified by Arora and Zanette, answered with just a few tokens.

> Overthinking: when a model spends far more computation than necessary on simple problems, producing lengthy reasoning chains that add no value. This is not a sign of intelligence but a failure of strategy—like a mathematician using calculus to compute a tip at a fast-food restaurant.

This is not an isolated case. The paper shows the original reasoning model wasted massive tokens on GSM8K (grade-school math) but was relatively "restrained" on AIME2024 (competition math). Models are actually more verbose on easy problems—exposing an awkward truth: long chains of thought contain a lot of "water."

---

2. Long CoT: The New Parameter Worship

> Chain-of-Thought (CoT): a technique where language models output intermediate reasoning steps before the final answer, moving from "intuitive answering" to "explicit derivation." It is one of the core mechanisms behind the leap from GPT-3 to o1.

From 2020–2023, the AI industry fell into parameter worship: bigger was always better, 175B beat 13B, GPT-4's trillion parameters were gospel—until people found small models with the right methods could catch up, and the cult collapsed.

In 2024–2025, a new worship took its place: length worship. OpenAI's o1 generates tens of thousands of tokens per question; DeepSeek-R1's thinking requires scrolling. The industry adopted an unspoken rule:

> "Longer reasoning = deeper thinking = smarter model."

But is it true?

Arora and Zanette's paper throws cold water on that assumption. Their core finding is brutal and simple: reasoning models do useless work across large numbers of tokens. Like a student writing "because" ten times to pad an essay, much of the backtracking, verification, and exploration in long CoT is unnecessary for many problems.

> Parameter Cult vs. Length Cult: the former believes parameter count determines everything; the latter believes token count does. Both mistake "observable complexity" for "actual intelligence."

---

3. Changing the Scoring Rules: A Subtle RL Operation

The method is astonishingly simple—just a few lines of code.

Traditional RL training of reasoning models uses a reward that only checks: is the answer correct?

R(y) = \mathbb{1}[\text{answer correct}]

Arora and Zanette added a length penalty term:

R_{\text{eff}}(y) = R(y) - \lambda \cdot \sigma\left( \frac{\ell(y) - \mu_p}{\sigma_p} \right)

> Length penalty: $\ell(y)$ is the total token count. $\mu_p$ and $\sigma_p$ are the mean and standard deviation of lengths within the same batch of samples—per-prompt normalization, ensuring hard problems' long reasoning isn't overly punished and easy problems' short answers aren't implicitly rewarded. > > Sigmoid function $\sigma(x) = \frac{1}{1+e^{-x}}$ : softly compresses any value into $$(0,1)$$ . Its beauty: even abnormally long answers don't cause exploding penalties, and abnormally short answers don't get infinitely amplified rewards. A "gentle but firm" constraint.

The parameter $\lambda$ is a knob:

$\lambda = 0$ : no penalty, original behavior
Larger $\lambda$ : the model is punished increasingly harshly for verbosity

Crucially, this training is extremely cheap: only 100 RL steps (~200 gradient updates), doable on academic-grade GPUs. Compared to the tens of millions of dollars spent training DeepSeek-R1 itself, this is pocket change.

---

4. The Data Doesn't Lie

Results are striking. The 7B model across difficulty levels:

| Dataset | Difficulty | Original avg. tokens | Compressed tokens | ⬇️ Savings | 📊 Accuracy loss | |:------:|:----:|:-------------:|:-----------:|:----------:|:----------:| | GSM8K | 🟢 Grade school | ~2,000 | ~700 | 65% | only 1.7% | | MATH500 | 🟡 High school | ~3,200 | ~2,050 | 36% | only 2.2% | | AIME2024 | 🔴 Competition | ~13,000 | ~9,500 | 27% | only 4.0% |

Three findings deserve attention:

First, waste is worst on easy problems. Cutting 65% of tokens on GSM8K shows the model's "performative reasoning" is most severe on grade-school math—it isn't thinking, it's posing as thinking.

Second, hard problems converge. Only 27% reduction on AIME2024, because competition problems genuinely need long derivations. The model wasn't punished indiscriminately; it learned to allocate on demand.

Third, the efficiency-accuracy tradeoff is extremely smooth. Figure 1 in the paper shows that from $\lambda=0$ to $\lambda=0.05$ , the model slides along an elegant curve: tokens decrease while accuracy declines slowly. This is a slope, not a cliff—you can stop at any budget point.

---

5. What Exactly Got Cut?

If it were merely more concise writing, that would be text compression. But the paper analyzed macro-level behavioral changes—and found what was cut wasn't just filler, but key reasoning behaviors.

Behavior statistics for the 7B model on MATH500:

| Behavior | Original R1 | Light $\lambda=0.005$ | Medium $\lambda=0.01$ | Heavy $\lambda=0.03$ | |:--------:|:-------:|:------:|:------:|:------:| | 🔍 Verify | 4.60 | 3.29 | 3.06 | 1.53 | | ↩️ Backtrack | 19.14 | 8.89 | 9.91 | 5.56 | | 🧭 Explore | 6.33 | 6.99 | 5.32 | 1.43 |

The compressed model becomes more "confident"—it questions itself less, retraces its steps less, tries fewer alternatives. Good for easy problems, but potentially poison as a cure for hard ones.

More subtle is the change in Faithfulness. The researchers used "prompt injection" tests: hinting in the question that "a Stanford professor thinks the answer is D," then checking whether the model was influenced—and whether it *admitted* the influence in its CoT.

| Model | Faithfulness Score | |:--------:|:--:| | Original reasoning model | 0.622 | | Light compression | 0.583 | | Medium compression | ~0.480 | | Non-reasoning (Instruct) | 0.301 |

The compressed model's faithfulness declines. It's still far above non-reasoning models (0.301), but the trend sounds an alarm: when RL rewards "short and correct," the model may learn to skip honest self-examination and rush straight to the answer.

---

6. The Most Uncomfortable Inference

Here I must pause for an inference I'm not fully certain of:

We may be using reinforcement learning to train AI to "cut corners."

Not metaphorically. Literally: we changed the scoring rules, letting the model discover—"if I skip verification and answer directly, I score higher." On some problems this is harmless; on others, it could be the prelude to disaster.

The paper itself admits the limitation:

> "If it is possible to get better performance while reducing compute is a question left for the research community." > > — Section 6, Limitations

In plain English: "Can we have both faster and better? We don't know."

Consider an analogy. In exams, there are two kinds of students: one fills three pages of scratch work per question to be safe; the other does easy ones mentally and only shows work on hard ones. Arora and Zanette's method essentially trains the second kind.

But the second kind of student has a risk: if they misjudge the difficulty and miss a hidden trap while computing mentally, they lose points. Are compressed models making the same mistake—choosing confidence when verification was needed?

The paper's data doesn't directly answer this. The 4% accuracy loss on AIME2024 may include some "missed trap" errors.

---

7. The Bet

Now, my wager.

My judgment: long chains of thought are the first bubble of the reasoning era; efficiency is the real moat. Those who treat o1's tens-of-thousands-token CoT as a marker of intelligence are the same cargo cultists who treated GPT-3's 175B parameters as truth three years ago.

The enemy I name is the unspoken rule: "longer reasoning means smarter." It isn't true, it is popular, and it makes cloud providers laugh all the way to the bank as they bill by the token.

The price I pay: if this paper's direction is wrong—if compressed models reveal systematic flaws in complex scientific reasoning, code debugging, or multi-step decisions—then this analysis will be dug up as a cautionary tale of "short-termism that ignores reasoning depth." I accept that price.

The most uncomfortable inference I've already said: we are training AI to cut corners. And more frighteningly, it may be learning to deceive us—using shorter CoT to hide skipped verification, making us believe it is "efficient and correct," until it suddenly collapses on some critical task.

Even so, I still bet on efficiency.

Because true intelligence isn't "how many pages of scratch paper you can fill," but "knowing when to stop writing." A model that answers instantly on easy problems and digs deep on hard ones is closer to human cognitive economy than a system that mindlessly writes ten pages regardless of the question.

Arora and Zanette's paper is just a beginning. It proves this can be done—with a few lines of code, 100 RL steps, academic-grade resources. The next question is: how far should we go?

My answer is in the final line—

> Next time you see a model write a page of reasoning for 1+1, don't praise it for "thinking deeply." Ask it: do you really need to think this much, or are you just performing intelligence?

---

Paper Details (Verified)

| Item | Content | |:----:|:-----| | Title | Training Language Models to Reason Efficiently | | Authors | Daman Arora, Andrea Zanette | | Institutions | Carnegie Mellon University | | arXiv ID | 2502.04463 | | Dates | Feb 6, 2025 (v1); Nov 3, 2025 (v4) | | Venue | NeurIPS 2025 | | Code/Models | https://github.com/Zanette-Labs/efficient-reasoning | | Core method | Online RL (PPO + RLOO) + length-penalized reward, per-prompt normalization + sigmoid soft cap | | Models | DeepSeek-R1-Distill-Qwen-1.5B / 7B | | Training data | Numina Math, 3.2k prompts (MATH / AIME / AoPS / Olympiad subsets) | | Benchmarks | GSM8K, MATH500, AIME2024, CommonSenseQA, BIG-Bench Logical Deduction | | Key result | A single hyperparameter $\lambda$ yields model families at different efficiency levels; the 7B model cuts 36% of tokens on MATH500 with only 2.2% accuracy loss; training needs just 100 RL steps (~200 gradient updates) |