Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

In May 2026, Chen et al. extracted and quantified search trees from LLM reasoning traces in Connect Four, revealing a deep characteristic of LLM planning: although models generate extensive deep look-ahead analysis in their chain-of-thought (CoT), their actual move choices are best explained by myopic models that ignore these deep nodes.

Key points

• Method: The authors parse natural-language CoT text, extract search trees (nodes = board states, edges = moves), fit computational models, and quantify structural features such as search depth, search breadth, and node expansion patterns.
• Surface vs. function: Reasoning models produce CoT that superficially resembles human expert look-ahead (e.g., "if I play column 3, opponent may respond column 4..."), but this deliberation does not constitute genuine deep planning.
• Core findings:
• Move choices are best explained by myopic models that ignore deep nodes.
• Search breadth (average branching factor), not depth, predicts win rate.
• Deep nodes serve a largely decorative role, unlike in human experts, whose performance is driven by deep search.
• Causal intervention:
• Full CoT → baseline move quality.
• Pruning deep segments (leaving only shallow) → move quality almost unchanged.
• Pruning shallow segments (leaving only deep) → significant degradation.
• Conclusion: shallow nodes are the decision-making core; deep analysis contributes near zero.

Interpretation: "performative thinking"

The proposed hypothesis is that deep CoT analysis is a statistical reproduction of "expert analysis patterns" from training corpora rather than a faithful record of actual computation:

• Pretraining acquires linguistic co-occurrence patterns — the textual form of deep analysis.
• SFT imitates human CoT — the form without the function.
• RLVR optimizes answer correctness and may reinforce shallow shortcuts.

Implications for test-time scaling

• Longer CoT ≠ deeper thinking; more test-time compute may be wasted if deep segments don't participate in decisions.
• Formal similarity of look-ahead text ≠ functional equivalence to planning.
• New evaluation dimensions proposed: actual decision depth (performance change after pruning different depth segments), structure–function consistency, and intervention robustness.

Relation to prior work

• Coupling Tax: the hidden cost of long reasoning chains — this study shows most of the long chain may not participate in decisions at all.
• 80/20 Rule on critical tokens: decisions may depend only on the first ~20% of CoT, with the rest being performative filler.
• Prefix Consistency: the pruning experiments act as depth-selective perturbations.

Limitations and future directions

• Generalization beyond Connect Four to chess/Go, mathematical proof, and program synthesis.
• Scale effects: do larger models plan more deeply? Reasoning-specialized vs. general architectures.
• Training interventions: deep-pruning penalties, mandatory citation of deep analysis in decisions, hierarchical attention mechanisms.
• Human–AI alignment: division of labor between myopic AI and far-sighted humans; risks of shallow planning in high-stakes settings.

Conclusion

The study establishes a form–function dissociation in LLM planning: CoT contains the form of deep look-ahead, but decisions are shallow-driven. Combined with causal evidence and a reusable search-tree extraction methodology, it calls for more careful evaluation of whether "more thinking" actually translates into "better decisions."

Paper details

• Title: Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning
• Authors: Sixing Chen, Ji-An Li, Saner Cakir, Sinan Akcali, Kayla Lee, Marcelo G. Mattar
• Affiliations: New York University, et al.
• arXiv ID: 2605.06840
• Date: 2026-05-07