Introduction
This is a Feynman-style walkthrough of the paper "Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization" (arXiv:2609.35643) by Huzi Cheng and Zhewei Zhang. Its central claim: a talking scratchpad may be the worst thinker of all.
The Setup: Five Exam Candidates, One Brain
The authors train five models from scratch on the same tiny GPTNeoX skeleton (4 layers, hidden size 256, 4 attention heads, FFN 768), identical in every way except the "thinking interface" used before answering:
- Direct: outputs the answer immediately, no intermediate steps.
- Chain-of-Thought (CoT): writes a full step-by-step proof along the shortest path before answering; supervised on both answer and intermediate steps.
- Pause-token: emits 6 identical placeholder tokens ("......") before answering—a form of 'fake thinking' known to help in prior work.
- Full-latent reasoning (Coconut-style): feeds the previous step's hidden state vector back as input, iterating K=6 times; it can still attend to the original prompt when answering.
- Bottleneck-latent: same recurrent latent thinking, but after thinking it can only access the question through the 6 latent vectors—forced to compress all useful information into them, like a closed-book exam with only 6 scratch sheets.
- Bottleneck-latent performs best, slightly ahead of Full-latent.
- Direct and Pause-token fall behind.
- CoT collapses to the worst performance—despite being the only model supervised on step-by-step proofs.
- Direct and Pause-token answers swing with candidate in-degree—they guess by local node statistics.
- CoT is biased too: even its first written step (choice of successor) follows the in-degree-minus-out-degree heuristic.
- L4H1: a clean address–content tracer—swapping the left side of "A is B" affects outputs via keys (address), the right side via values (content); it retrieves successors of currently reachable nodes each iteration.
- L4MLP + residual stream: not a standalone additive part but a filter embedded in the residual path, integrating retrieved graph relations into the latent state z_t.
- Multi-head candidate matching: H1 writes candidate info into the latent state; H2 acts as the final readout, matching candidates against the reachable set.
- Huzi Cheng, Zhewei Zhang. *Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization*. arXiv:2609.35643 [cs.AI], 2026-09-28. https://arxiv.org/abs/2609.35643
All share the same data, training budget, and hyperparameters (AdamW, lr 4×10⁻⁴, batch 256, BF16).
The Exam: ProsQA-Ext
The task is a multi-hop graph reachability problem derived from ProsQA: given premises forming a DAG and a root r plus two candidates, which candidate is reachable from r? Difficulty is controlled by hop count H. Training covers H=3–6 (400k problems); the OOD test covers H=7–12—reasoning chains never seen in training. The generator carefully eliminates surface statistical cues, so appearance-based guessing yields only ~50% accuracy.
Same ID Scores, Different Fates
In-distribution, all models score well. But on OOD deep chains:
ID performance is a report card that measures whether you got it right, but not why.
Cheating with 'Feng Shui': Statistical Shortcuts
RSA (representation similarity analysis) shows the two latent models internally track forward propagation of a reachable-set frontier—a clean diagonal alignment quantified by a "Diagonality" metric. Direct and Pause-token show no such alignment, yet score well in-distribution. Why? The paper constructs paired datasets isolating residual statistical cues (correct-answer nodes tend to have lower in-degree):
The latent models are nearly immune: they aren't guessing, they're searching.
Causal Proof: A Soft for-Loop in the Head
State-transplant interventions confirm causality: swapping an edge at depth d between twin graphs and grafting the paired latent state at step t flips the answer in an "upper-triangle" pattern—perturbations at shallow depth act early, deep ones late. The computation is causal, not decorative. The "soft loop" hypothesis (search driven by content, not strict one-step-per-hop) is confirmed: only Bottleneck-latent is stable when K changes from 6 to 5 or 7.
Circuit Dissection: The 8-Component Search Team
Greedy pruning of 20 recurrent components leaves just 8 components preserving 91.9% output consistency (0.837 R²), generalizing to unseen 7–12 hop problems (90.9%), versus 52.7% when all 8 are removed:
The full algorithm—latent state maintains the reachable set; L4H1 expands it; L4MLP filters and integrates; heads match candidates—emerged purely from end-to-end answer supervision, with no CoT demonstrations, no RL, no curriculum.
Why the Silent Thinker Goes Farther
Written text ≠ thinking: in CoT, language became the best canvas for hiding shortcuts. Pause-token's interior is as empty as Direct's—extra time alone doesn't produce thought; thought comes from interfaces that let information flow between steps. The bottleneck didn't kill reasoning—it forced it: precious working memory can hold shortcuts or algorithms, not both. And recurrence is a natural while-loop mold; an algorithm fits it, and any depth is just the same loop turned a few more times.
Takeaways and Limitations
The authors honestly note limits: generalization to large pretrained models and natural tasks remains open, as does whether explicitly looped Transformers share the same mechanism. But the methodology is scale-independent:
1. Behavior first, mechanism second—OOD testing is the Stroop test for models. 2. Control everything but the thinking interface. 3. Mechanisms must pass causal tests—RSA correlation, state transplantation, pruned-circuit replication, down to key/value roles of specific heads.
Engineering intuition: if you want a model that truly reasons, give it a non-verbal working memory, block the back doors, and let it figure things out in silence.