This article explores two complementary theories of intelligence: the Geometry of Thought (modeling reasoning as motion through a curved conceptual space) and the Greedy Trap (the systematic failure of locally optimal decision strategies). Their unification suggests that greedy failures are non-geodesic trajectories, and that escaping them requires exploiting global geometric structure.
Key points
1. Cognitive Geometry (Geometry of Thought)
- Intellectual lineage: The framework draws on three traditions — Gärdenfors' conceptual space theory, manifold learning in machine learning (t-SNE, UMAP, the manifold hypothesis), and hippocampal cognitive-map research in neuroscience. The most directly relevant recent work is Alpay & Senturk (Jan 2025), "The Geometry of Thought," which links Transformer attention to tropical geometry. (Note: a Chinese "Riemannian manifold framework" lab mentioned in some discussions could not be independently verified in academic literature.)
- Riemannian conceptual space: High-dimensional data lie near a low-dimensional manifold. Key elements:
- Metric tensor \(g_{\mu\nu}\): position-dependent distances model context-dependent semantic similarity (e.g., "king − man + woman ≈ queen" analogies, circular color hue topology).
- Curvature \(R_{\mu\nu\rho\sigma}\): high-curvature regions correspond to cognitively complex concepts; positive curvature suggests clustering/fast generalization, negative curvature fine-grained differentiation.
- Geodesics \(\gamma(t)\): optimal reasoning paths; the geodesic equation \(\frac{d^2\gamma^i}{dt^2} + \Gamma^i_{jk}\frac{d\gamma^j}{dt}\frac{d\gamma^k}{dt} = 0\) distinguishes deliberate, curvature-aware reasoning from intuitive straight-line jumps.
- Tropical geometry and Transformers: In the high-confidence limit (\(\beta \to \infty\)), softmax attention
- AlphaGeometry (DeepMind): Solved 25/30 IMO geometry problems, near gold-medal level, via a dual system — a neural language model (Kahneman "System 1", proposing auxiliary constructions) plus a symbolic deductive engine DDAR ("System 2", rigorous verification), trained on 1 billion synthetically generated diagrams without human demonstrations. Its conceptual (non-coordinate) reasoning illustrates the neural-symbolic-geometric fusion paradigm.
- Experimental evidence: Triadic comparison tasks report ~3.7% violation of the triangle inequality (weak metric structure), significant deviation from zero curvature (\(t=4.89, p<0.001\), mean Gaussian curvature \(0.152 \pm 0.031\)), and curvature–cognitive-load correlations of \(r = 0.42\) (reaction time) with 72.3% cross-validated accuracy predicting subjective learning difficulty.
- Fast/slow thinking formalized: System 1 corresponds to high-temperature gradient flow \(\frac{dx}{dt} = -\nabla_g V(x) + \sqrt{2T}\xi(t)\) on a cognitive potential \(V\); System 2 to curvature-aware, momentum-based, or global planning strategies. Riemannian gradient descent (\(x_{t+1} = \text{Exp}_{x_t}(-\eta\, \text{grad}\, V(x_t))\)) adapts step size to the local metric.
- Classical algorithms: Greedy Best-First Search ignores path cost \(g(n)\) and fails under heuristic deception; A* (\(f = g + h\)) retains guarantees when \(h\) is admissible. TSP nearest-neighbor degrades linearly on star-shaped instances; the 0-1 knapsack value-density greedy fails because indivisibility breaks the greedy-choice property. Greedy methods succeed only when optimal substructure and the greedy-choice property hold (formalized by matroid theory) — conditions most AI problems (theorem proving, planning, creative tasks) do not satisfy.
- LLMs as greedy agents:
- Low action coverage: Gemma2 27B explores only ~60–65% of actions in 10-armed bandits, saturating after ~10 steps, with linearly growing cumulative regret versus the logarithmic regret of UCB.
- Frequency bias: models favor statistically frequent actions, compounding algorithmic greediness.
- Knowing–doing gap: Gemma2 27B correctly identifies the UCB-optimal arm 87% of the time but picks the globally optimal arm only 21% (58% greedy choices); correct computation leads to optimal action only 24% of the time. Knowledge and decision-making appear architecturally separated — not fixable by simple fine-tuning.
- CoT limitations: insufficient coverage of the latent space, local lock-in, length limits (hundreds of tokens vs. thousands needed), and self-bootstrapping error contamination; in the tropical view, limited depth equals limited Bellman-Ford iterations.
- Scale does not automatically help: larger models show more elaborate surface exploration but similar substantive greediness; exploration–exploitation balance is non-monotonic across model sizes.
- Human parallel: Kahneman–Tversky heuristics (availability, representativeness, anchoring) show the same structural short-sightedness; educational studies find over a third of students apply greedy strategies to minimization problems and fail on adversarial configurations. The key human advantage is metacognition — the ability to detect and switch to slow, global reasoning.
- Greedy trajectories approximate the geodesic equation by dropping curvature terms (\(\frac{d^2\gamma^i}{dt^2} \approx 0\)), which is acceptable in flat regions but diverges severely in high-curvature ones. Deviation types include metric misestimation, curvature neglect, topological obstacles, and rigid step sizes.
- Landscape geometry dictates behavior: shallow local minima, saddle-rich terrain (dominant in high dimensions), flat plateaus, and steep canyons each require different escape strategies (noise, momentum, preconditioning, second-order information).
- Overcoming the trap means exploiting global geometric information: curvature estimates (adaptive step sizes), spectral and topological invariants (persistent homology, connectivity analysis), geodesic distances, and curvature-aware exploration where uncertainty is treated as curvature and information gain as distance.
- Geometric intelligence as a third paradigm beyond symbolism and connectionism: symbols as constrained submanifolds, learning as manifold function approximation, reasoning as geodesic tracking.
- Improved reasoning: explicit token-graph construction, A*/Bellman-Ford-style geodesic search instead of greedy decoding, hierarchical planning, and dynamic per-problem depth allocation.
- Knowing–doing alignment: shared manifold representations for knowledge and action, geometric regularization keeping actions near the cognitive geodesic, counterfactual training, and symbolic verification loops.
- Architecture directions: curvature-aware continual learning (manifold extension rather than overwriting, to avoid catastrophic forgetting), multi-agent coordination via shared metrics and Fréchet-mean consensus, and geometry-based value alignment.
- The article notes speculative elements (golden-ratio "cognitive quasicrystals", geometric models of consciousness via fiber bundles and prediction-error feedback) that remain unverified and require further empirical support.
\[\text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V\]
converges to tropical form:
\[\text{TropicalAttention}(Q,K,V)_{ij} = \max_k (Q_i \cdot K_k + V_{kj})\]
This is exactly a Bellman-Ford update: each Transformer layer performs one shortest-path relaxation step, and Chain-of-Thought reasoning is explicit shortest-path search over a latent token graph. CoT failures can be diagnosed as graph disconnection, negative cycles, or insufficient iterations. The limitation: the equivalence is strict only at \(\beta \to \infty\), while deployed models run at finite temperature.