The Illusion of Thinking: LLM Performance Collapse and Deterministic Loops on Towers of Hanoi

Forum topic · ✨步子哥 · 2025-11-12

Summary

This article analyzes Apple's controversial research paper 'The Illusion of Thinking' and follow-up analyses examining why large reasoning models (LRMs) fail on the Towers of Hanoi puzzle. Key findings: model performance does not degrade gracefully with problem complexity but collapses catastrophically past a threshold (around 5-6 disks), with success rates dropping from near-perfect to near-zero. A three-stage pattern emerges: at low complexity standard LLMs outperform reasoning models (which may 'overthink'), at medium complexity reasoning models excel via chain-of-thought planning, and at high complexity all models collapse. Counterintuitively, models reduce token usage and effectively 'give up' on hard problems despite ample resources, and fail even when given explicit correct recursive algorithms. Failure manifests as 'deterministic loops' — repeated, legal but futile move sequences — suggesting LLMs perform sophisticated pattern matching over training data rather than genuine logical deduction. The article explores internal mechanisms including representation collapse, attention fixation, autoregressive sampling greediness, and the Transformer architecture's lack of native recursion. Agentic frameworks (stepwise prompting, multi-agent dialogue, Modular Agentic Planner) that externalize state management delay but do not eliminate failure at higher complexity.

Key points

This post analyzes Apple's research paper *The Illusion of Thinking* and related follow-up work, focusing on why large reasoning models (LRMs) collapse on the Towers of Hanoi puzzle.

The performance collapse

LRMs do not degrade gracefully with problem difficulty. Success rates stay near-perfect for 3-4 disks, then plunge to near zero at around 5-6 disks — a sudden, catastrophic failure rather than gradual decline.
Towers of Hanoi is an ideal testbed: deterministic rules, controllable complexity (minimum moves = 2^n - 1), and unlikely to be memorized wholesale for larger n.
The follow-up paper *Rethinking the Illusion of Thinking* confirms that even with stepwise prompting and agentic collaboration, models still fail at moderate complexity increases (~8 disks).

Three-stage performance model

| Stage | Complexity | Best performer | Behavior | | --- | --- | --- | --- | | 1 | Low | Standard LLM | Direct, efficient; LRMs may 'overthink' | | 2 | Medium | Reasoning model (LRM) | Multi-step planning via chain-of-thought | | 3 | High | Complete collapse | All models fall to near-zero success |

Counterintuitive behaviors

Reduced reasoning effort: near the capability limit models consume the most tokens; beyond it, they dramatically *reduce* output length — an 'early abandonment strategy.'
Failure despite given algorithms: even supplied with a correct recursive algorithm, models cannot maintain the recursion depth and state tracking needed for high complexity.

Deterministic loops: the core failure mode

Failures are not random. Models fall into fixed, legal, futile move sequences — cycling among valid moves without converging toward the goal state.
In agent frameworks where the environment validates all moves (so no illegal moves occur), models still loop indefinitely, showing that rule-compliance is insufficient without goal-directed global planning.
Unlike humans who backtrack after errors, models mechanically repeat high-probability moves, lacking true goal orientation.

Root cause: pattern matching, not logical deduction

Research on Transformer multi-step reasoning suggests models reduce inference to linearized subgraph matching — matching current states against computation patterns seen in training.
Solutions for 3-4 disks are abundant in training data; full solution paths for 6+ disks (2^n - 1 steps) are sparse or absent, forcing unreliable stitching of fragments.
Models show failure, not adaptation, when facing novel structural complexity — they cannot derive recursive solutions from rules or invent new heuristics.

Internal mechanism hypotheses

Attention fixation: on hard instances, attention collapses onto familiar but irrelevant patterns (e.g., fixating on the largest disk), missing the auxiliary-peg steps needed for a solution.
Representation collapse: per the SEQ-VCR line of work, intermediate representation diversity shrinks on complex tasks, so logically distinct board states become indistinguishable in deeper layers.
Greedy autoregressive sampling: models prefer statistically 'safe' moves, lacking exploration and getting trapped in local optima.
No native recursion: Transformers must linearize the whole solution in a single forward pass; they can 'simulate' recursion from examples but cannot truly 'execute' a call stack.
Verifying these hypotheses directly is difficult because model internals remain a black box; current analyses infer mechanisms from external behavior.

Agentic framework design

The framework externalizes state management: an environment module maintains the Hanoi state and feeds the model a clear textual description each turn; the LLM acts purely as a policy/move generator. This ensures failures stem from reasoning itself, not memory errors.
Interaction loop: observe state → think → output a move → environment validates/executes → new state, until solved, looped, or step limit reached.
Tested interaction modes:
Stepwise prompting — still yields deterministic loops at high complexity.
Agentic dialogue (planner/executor roles, debate) — collaboration does not prevent infinite loops at sufficient complexity.
Modular Agentic Planner (MAP) — decomposes planning into conflict monitoring, state prediction, evaluation, and coordination modules; performs well on 3-4 disks (reported in *Nature Communications*), but effectiveness at higher complexity remains unverified.
A proposed experiment ('Hanoi Loop') would log all interactions and run automated loop detection (e.g., sliding-window checks for repeated move subsequences), quantifying loop length, frequency, and the disk count at collapse to empirically test the pattern-matching hypothesis.

Conclusion

The evidence suggests current LLM 'reasoning' is a hybrid: memory-driven at low complexity, planning-like at medium complexity, and fundamentally fragile at high complexity. Deterministic looping under ample resources supports the view that LLMs are sophisticated pattern matchers rather than generalizable logical reasoners — the core of the 'illusion of thinking.'