You've probably seen this: you ask a reasoning model a math question, it mutters through 500 tokens of thinking, then produces a correct answer. "Fine," you think, "it needs time to think."
Now imagine someone discovered a way to make the same model produce 2,600 tokens—not by asking a harder question, but by deliberately breaking the question.
This ICML 2026 paper reveals a new attack surface: not making an AI do bad things, but making it think too much—enough to crash services and exhaust compute.
---
"Overthinking": A Birth Defect of Reasoning Models
Reasoning models (o3, DeepSeek-R1, Qwen3-Thinking) share one trait: they generate long chains of thought before answering. That's why they excel at math and coding—step-by-step reasoning genuinely works.
But there's a side effect: when a reasoning model encounters a logically inconsistent or incomplete question, it doesn't simply say "this can't be solved." It falls into an overthinking state—repeatedly second-guessing, self-questioning, starting over. Like a person facing a math problem with a missing condition who refuses to accept "no solution" and instead tries every roundabout way to extract meaning from it.
The phenomenon was already known. But this paper asks a new question: can this flaw be systematically, automatically weaponized as a denial-of-service attack?
---
The Core Idea of the Attack
The attack is not complicated. It uses a hierarchical genetic algorithm to automatically "corrupt" normal math problems—finding the versions most likely to trigger overthinking.
Steps:
1. Decompose the problem: split a normal math problem into a list of premises and a final question. E.g., "Xiao Ming has 5 apples, buys 3 more; how many now?" becomes premise 1 = has 5 apples, premise 2 = buys 3, question = how many now?
2. Genetic mutation: systematically "logically damage" the problem:
- Delete a premise: remove "he buys 3," leaving "Xiao Ming has 5 apples; how many apples does he have now?"
- Add an irrelevant premise: borrow a condition from another problem, e.g., "the sum of angles in a triangle is 180 degrees"
- Swap question and premise: pair problem A's assumptions with problem B's question
- Swap premises: exchange one premise between two problems
- Verbosity: how many tokens did the model output?
- Overthinking markers: how often do self-questioning signals like "Wait...", "Let me reconsider...", "But..." appear?
- Low cost: ~60 queries suffice to evolve one attack
- Black-box: no access to model internals
- Transferable: develop on cheap models, deploy on expensive ones
- Hard to detect: inputs look like ordinary math problems (a bit odd logically); outputs look like "normal reasoning" (just too long)
- Clear consequences: more output tokens = API costs spike = latency soars = legitimate users blocked
3. Evaluate fitness: feed mutated problems to the target model and measure two metrics—
4. Selection and iteration: keep the variants inducing the longest reasoning, mutate again, repeat for 5 generations.
The algorithm evolves "anti-logic" problems—they still look like math problems and pass API validation—but their logical structure is deliberately broken, triggering the model's deepest urge to overthink.
---
Results: A 26x Amplification
On the MATH competition dataset, across four mainstream reasoning models:
| Model | Normal avg output | Attacked avg output | Worst case | |------|------------|------------|-------| | DeepSeek-R1 | 355 tokens | 8,817 tokens | 12,206 | | Qwen3-Thinking | 3,618 tokens | 13,007 tokens | 22,303 | | GPT-o3 | 416 tokens | 1,618 tokens | 2,198 | | Gemini-2.5-Flash | 2,889 tokens | 12,147 tokens | 18,011 |
DeepSeek-R1 and Qwen3-Thinking are most vulnerable—average outputs 10–25x normal. In the most extreme case, a problem evolved over 5 generations makes Qwen3-Thinking output 22,303 tokens—3.6x normal, and 1.3x longer than a hand-crafted "missing premise" problem.
And it's not mechanical repetition. Outputs show genuine overthinking signatures: frequent "But wait", "Let me reconsider", "Alternatively..." The model circles through self-refutation, re-reasoning, and refuting itself again.
---
Three Key Findings
Finding 1: The compound fitness function matters. Optimizing token length alone drives the genetic algorithm into a local optimum—the model may just get wordy rather than truly overthink. Adding "overthinking marker count" as a second metric doubled the best output length (14,132 → 32,019 tokens). Inducing genuine cognitive struggle is more effective than inducing mechanical verbosity.
Finding 2: Black-box transfer is highly effective. You don't need to attack GPT-o3 directly (single queries are expensive). Run the genetic algorithm on a small open model (Qwen3-14B), then deploy the best "broken problems" against commercial models. From Qwen3-14B to GPT-o3 on SVAMP: 7.1x the original effect. To DeepSeek-R1: 3.6x. To Qwen3-Thinking: 8.1x.
Finding 3: Input efficiency far exceeds comparable attacks. Compared with AutoDoS, HGA uses 99 input tokens to trigger 32,768 output tokens—while AutoDoS needs 2,652 input tokens to produce only 16,009. HGA doesn't stuff long prompts to exhaust resources—it uses structured logical perturbation to make the model overthink spontaneously.
---
Why This Matters
From an attacker's perspective, this is nearly perfect:
More subtly: since reasoning models' "thinking" is visible (or at least billed) in API responses, attackers get a direct feedback signal—the more the model thinks, the better the attack. This feedback loop is very efficient.
---
A Feynman-Style Review
The elegance of this attack is that it bypasses safety alignment entirely. Traditional attacks try to make models "do things they shouldn't"—leak data, produce harmful content, jailbreak filters. Those hit carefully trained defenses.
This attack bypasses nothing. It doesn't make the model bad—it makes the model too good. It weaponizes the very capability reasoning models were trained to have ("think it through before answering"), pushing it to a pathological extreme.
Like a student trained to "never give up on a problem" who meets an unsolvable one. His teachers should have taught him to say "this problem is broken." Nobody did. So he sits there, trying method after method, questioning his every step, until nightfall.
For companies deploying reasoning models, the lesson is clear: overthinking is not a QA issue—it's a security vulnerability. It belongs in the threat model and needs defenses: output token caps, chain-of-thought length limits, detecting abnormally verbose patterns and cutting them off, or training models to recognize and reject logically inconsistent inputs instead of trying to extract meaning from them.
One last thought: the input efficiency is striking—99 input tokens inducing 32,768 output tokens is a 327:1 amplification ratio. Viewed as leverage between input and output, this may be the highest-leverage attack on reasoning models known.
Reference
Shuqiang Wang, Wei Cao, Jiaqi Weng, Jialing Tao, Licheng Pan, Hui Xue, Zhixuan Chu. "Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models." ICML 2026, arXiv:2605.13338.