Give DeepSeek-R1 an AIME competition math problem. It starts "thinking"—outputting a long stream of reasoning tokens. You wait 10 seconds, 20 seconds, 40 seconds, and it's still thinking. You begin to wonder: is it actually approaching an answer, or is it already lost?
A paper by Renuka Oladri et al., *Token Budget Saturation and Mechanistic Early Detection of Reasoning Non-Convergence in Chain-of-Thought Models*, offers a surprising answer: the fate of a problem is largely sealed by around the 150th token of reasoning.
A Bimodal Distribution: Instant Success or Endless Struggle
The researchers fed three math benchmarks to DeepSeek-R1-Distill-Qwen-7B: GSM8K (elementary), MATH-500 (high school), and AIME 1983–2024 (competition level). Using a budget-forcing mechanism to cap thinking tokens from 256 to 10,000, they measured how accuracy changed.
The resulting curve is striking:
GSM8K and MATH-500: 256 tokens reach 95% of unrestricted accuracy. Simple problems simply don't need long reasoning.
AIME is a completely different story. It shows an extreme bimodal distribution:
- 56.5% of problems "naturally converge"—the model finishes reasoning within 10,000 tokens, with 96.5% accuracy
- 43.5% of problems "do not converge"—reasoning runs to the 10,000-token cap without a conclusion, with only 11.5% accuracy
- At token 150, activations from layer 20 yield AUC 0.608 (±0.080, 5-fold cross-validation)
- Even at token 50—reasoning barely begun—the signal is already reliably above chance
- Activation probes outperform behavioral baselines (based on token entropy and repetition statistics) at 8 of 9 checkpoints, with an average AUC gain of +0.035
- Stop non-converging problems early, saving 9,800+ tokens of compute
- Continue converging problems, preserving accuracy
- Or switch strategies for non-converging ones—new prompt, new tools, or fallback to retrieval
- Easy problems (GSM8K/MATH-500) saturate at 256 tokens; extra tokens are pure waste
- Hard problems (AIME) are bimodal—extra tokens only help the subset that would converge anyway, and do nothing for the non-converging rest
- AUC 0.608 is not a strong signal—better than random, far from practical
- With n=200, the permutation test only reaches p=0.063, missing the 0.05 line
- Tested on a single model (DeepSeek-R1-Distill-Qwen-7B); generalization unknown
- There is an engineering gap between "early detection" and "actual early exit"—probes trained offline may not run feasibly in real-time inference
The gap is astonishing: converging problems score 96.5%; non-converging ones 11.5%. For a given problem, the model either knows it—pushing through to the end—or doesn't, and grinding for ten thousand tokens changes nothing.
Moreover, this bimodal split is only weakly correlated with problem difficulty (r² ≈ 0.186). Hard problems aren't necessarily non-converging; some hard problems are solved instantly, while some medium ones drag on. Fate isn't fully determined by difficulty.
Early Detection: Seeing the Fate at Token 150
Even more surprising is the second finding. The researchers asked: since outcomes are bimodal, can they be detected early in the reasoning process?
They inserted forward hooks during inference to capture hidden-layer activations, then trained linear probes—simple logistic regression classifiers reading a single layer at a single position—to predict whether a problem would ultimately converge.
Results:
In other words: whether the model "knows it can solve the problem" is already partly encoded in its internal representations by the 50th token—before the reasoning chain is complete, its neurons are quietly telling you the ending.
A permutation test with 100,000 permutations gives p=0.063—slightly above the conventional 0.05 threshold, suggesting the sample size (n=200) is a bit small, but the signal direction is clear.
What This Means: Adaptive Compute Allocation
If a problem's fate can be detected early, something very practical becomes possible: early exit.
Current CoT reasoning gives every problem the same token budget—easy problems waste compute, hard ones don't get enough. With an early detector judging at token 150 that a problem is unlikely to converge, you could:
This is the "adaptive compute allocation" in the paper's framing. Not every problem is worth grinding for ten thousand steps.
"More Thinking" Is Not a Panacea
The study directly challenges the default assumption behind CoT scaling—"more thinking tokens means better results":
This aligns with prior findings that CoT gains are non-linear with inflection points. But this work goes further: the inflection point varies by problem, and the outcome is encoded in representations early.
Honest Assessment: Weak Signal, Small Sample
Limitations must be noted:
An Analogy: In Exams, You Either Know It or You Don't
You've probably had this exam experience: looking at a problem, you either "have an idea" or you don't. Problems with an idea, you derive all the way through, likely correctly; problems without one, you can grind on scratch paper for half an hour and get nowhere.
DeepSeek-R1's CoT reasoning exhibits exactly this bimodal fate of "you either know it or you don't." And its internal representations—those hidden-layer activations—already "know" whether it has an idea from the very start of reasoning; you just can't see it from the outside.
This raises a deeper philosophical question: is the model's "knowing it can't solve this" the same kind of knowing as a human's? When a linear probe can read "I can't crack this problem" from neuronal activations, is that a prototype of metacognition?
Maybe. But for now, at least, we can use it to save compute.
---
Paper: Renuka Oladri, Niveda Jawahar, Abdirisak Mohamed. *Token Budget Saturation and Mechanistic Early Detection of Reasoning Non-Convergence in Chain-of-Thought Models.* arXiv:2607.21433, 2026. Link: https://arxiv.org/abs/2607.21433