The Illusion of Thinking: Analyzing LLM Performance Collapse and Deterministic Loops in the Tower of Hanoi

Forum topic · ✨步子哥 · 2025-11-12

Summary

This article analyzes Apple's controversial paper 'The Illusion of Thinking' and its follow-up critiques, focusing on how large reasoning models (LRMs) fail catastrophically on Tower of Hanoi puzzles. Using Hanoi as a controllable-complexity testbed, experiments show a three-stage pattern: at low complexity (e.g., 3 disks) standard LLMs outperform reasoning models; at medium complexity (4-5 disks) LRMs excel via chain-of-thought planning; at high complexity (6+ disks) all models collapse to near-zero success. Counterintuitively, models reduce token usage and 'give up' when problems exceed their capability, even when provided with an explicit correct recursive algorithm. The failure mode is characterized as deterministic loops: models repeat fixed, legal-but-useless move sequences rather than exploring new strategies. Root causes traced include Transformers' reduction of multi-step reasoning to linearized subgraph matching, heavy dependence on training-data patterns (2^n-1 step solutions become sparse at scale), representation collapse in intermediate layers, greedy autoregressive sampling, and the absence of native recursion mechanisms. The article also describes agentic frameworks—stepwise prompting, multi-agent dialogue, and Modular Agentic Planner (MAP) designs—that externalize state management, and proposes a 'Hanoi Loop' experiment with loop-detection to empirically verify the pattern-matching hypothesis.

Key Points

Apple's paper *The Illusion of Thinking* and the follow-up *Rethinking the Illusion of Thinking* reveal that large reasoning models (LRMs) do not degrade smoothly with puzzle difficulty—they collapse abruptly past a complexity threshold, suggesting pattern matching rather than genuine, generalizable logical reasoning.

Three-Stage Performance Pattern on Tower of Hanoi

The Tower of Hanoi is an ideal testbed: deterministic rules and quantifiable complexity (minimum moves = 2^n − 1). Findings:

| Stage | Complexity | Best Performer | Behavior | | :--- | :--- | :--- | :--- | | 1 | Low | Standard LLM | Direct, efficient; LRMs may "overthink" | | 2 | Medium | LRM | CoT-based multi-step planning succeeds | | 3 | High | Total collapse | All models crash to near-zero success |

Success rates drop sharply around 5-6 disks (even with improved prompting/agentic setups, models struggle around 8 disks).
Counterintuitively, models reduce output token usage on problems beyond their ability—an "early surrender strategy" implying an internal difficulty-assessment signal.
Models fail even when given an explicit correct recursive algorithm, showing they cannot sustain the recursion depth and state tracking required.

Deterministic Loops: The Core Failure Mode

When stuck, models fall into deterministic loops: repeating a fixed, legal-but-useless sequence of moves (e.g., shuttling disks between two pegs), reproducible across runs. Key observations:

Every individual move is legal; the failure is a failure to converge toward the goal, not rule violations.
Unlike humans who backtrack after errors, models mechanically repeat high-probability moves—lacking global, goal-directed navigation.

Root Causes: Pattern Matching, Not Logical Deduction

1. Linearized subgraph matching: Transformers perform multi-step reasoning via attention-based "matching operations," which fail when state spaces grow exponentially and learned subgraphs no longer compose correctly. 2. Training-data dependence: Solutions for 3-4 disks are ubiquitous in training data; complete 2^n − 1 step sequences for larger n become sparse, so stitching partial patterns easily derails into loops. 3. No adaptation to new structure: models cannot derive recursion from first principles or learn new heuristics from trial and error. 4. Representation collapse: per SEQ-VCR research, intermediate-layer representations lose diversity at depth, making distinct board states indistinguishable. 5. Greedy autoregressive sampling: preferring high-probability "safe" moves over counterintuitive but necessary ones, causing local-optima traps. 6. No native recursion: Transformers cannot pause a task, solve an n−1 subproblem, and restore a call stack—they can only "simulate" recursion from learned examples.

Agentic Framework Design

To isolate pure reasoning from memory burdens, experiments externalize state management: the environment maintains the peg/disc configuration and feeds the model a clean state description each turn; the model only selects the next move. Interaction modes compared:

Stepwise prompting: observe → think → act → feedback loop; models still loop deterministically, proving the deficit is reasoning itself, not memory.
Agentic dialogue: planner/executor multi-agent collaboration still ends in infinite loops at high complexity.
Modular Agentic Planner (MAP): decomposes planning into task decomposition, prediction, evaluation, and coordination modules; excellent on 3-4 disks (published in *Nature Communications*), but effectiveness at higher complexity remains unverified.

Proposed Experiment: "Hanoi Loop"

A systematic protocol logging every prompt, model move, and environment response, with automated loop detection (sliding-window detection of repeated move subsequences) to quantify loop length, frequency, and onset disk-count—providing empirical evidence for the pattern-matching hypothesis.

Conclusion

The collapse is not simple "lack of intelligence" but a fundamental architectural boundary: attention fixation on familiar patterns, representation collapse, and the absence of recursion and goal-driven search. Current LLM "reasoning" is a hybrid—memory at low complexity, partial planning at medium complexity, and brittle failure at high complexity—supporting the characterization of the phenomenon as an "illusion of thinking."

*Reference: Apple, "The Illusion of Thinking"; "Rethinking the Illusion of Thinking"; "SEQ-VCR: Preventing Collapse in Intermediate Representations".*