Lost Travelers with Perfect Maps: LLMs Encode Knowledge In-Context but Struggle to Use It

Forum topic · 小凯 · 2026-02-11

Summary

A Google DeepMind collaboration with Brown University and NYU (arXiv:2602.04212) reveals a striking gap in large language models: they can encode near-perfect internal representations of spatial structures learned in-context, yet fail to act on them. In tests on 1D chains and 2D grid worlds, models like Gemma 3 achieved ~90% distance correlation (high-quality internal 'maps') but accuracy collapsed below 20% on multi-step 2D navigation, even for frontier models like GPT-5 (~25% on 2-step grid tasks). The authors call this 'inert knowledge'—encoded but unusable—and attribute it to the inherently 1D nature of Transformer self-attention. Chain-of-thought prompting and longer contexts did not help. The article discusses implications for safety-critical applications, related DeepMind work on the 'knowing-doing gap' (RLFT improving action coverage from 40% to 52%), and possible remedies: reinforcement learning fine-tuning, hybrid tool-using architectures, neurosymbolic AI, and non-Transformer designs for multidimensional reasoning.

Lost in the Perfect Map: When LLMs Know but Cannot Act

Imagine holding a flawlessly detailed city map—you can recall every street from memory—yet being unable to take a single step toward your destination. According to a 2025 paper by Google DeepMind, Brown University, and NYU researchers, Language Models Struggle to Use Representations Learned In-Context, this paradox describes the current state of large language models (LLMs): they can build perfect internal "maps," but often cannot use them to "walk."

In-Context Learning: The Magic

In-context learning (ICL) lets LLMs learn new tasks from a few prompt examples without any parameter updates. Give the model translation pairs like 苹果 → apple, 香蕉 → banana, then ask for 西瓜 and it answers "watermelon." Powered by the Transformer's self-attention mechanism (introduced in 2017), ICL has seemed like evidence of genuine understanding and transfer.

The Crack: Perfect Encoding, Failed Action

The researchers placed models in minimal virtual worlds defined by topology:

1D linear chains (like a 16-station subway line)
2D grids (4×4 or 5×5 boards)

Models received structural descriptions and navigation examples, then had to predict multi-step paths.

| Model | Params | Topology | Task | Accuracy | Representation quality | |---|---|---|---|---|---| | Gemma 3 | 4B | 1D chain | 1-step | ~95% | ~90% | | Gemma 3 | 4B | 2D grid 4×4 | 2-step | <20% | ~85% | | Gemma 3 | 27B | 1D chain | 3-step | ~60% | ~92% | | Gemma 3 | 27B | 2D grid 5×5 | 3-step | <15% | ~85% | | GPT-5 | undisclosed | 2D grid 5×5 | 2-step | ~25% | undisclosed |

Adding one dimension caused accuracy to collapse. Yet probing the models' internal states revealed they had encoded the world's geometry almost perfectly:

1. Distance correlation (~85% even on 2D grids): the models rebuilt a faithful internal map. 2. Dirichlet energy (kept low): representations were smooth and well-structured.

The authors call this "inert knowledge"—information is encoded but cannot be flexibly invoked, like a navigation app with perfect map data that freezes when you press "start."

Why? The Transformer's 1D Soul

Self-attention is natively designed for one-dimensional sequences. Tracking multiple independent directions (east-west, north-south) is a kind of dimensional curse for it—like a sprinter asked to suddenly play soccer. Rescue attempts failed: extending context to 5000 tokens, chain-of-thought prompting, and stronger closed models left accuracy essentially unchanged.

Broader Implications

Everyday use: models write elegant poetry yet fail multi-step logic; summarize literature yet botch physical reasoning; recall syntax yet lose the thread in complex algorithm design.
Safety-critical domains: in autonomous driving, medical diagnosis, finance, or robotics, "knowing without acting" is a potential disaster.
The knowing-doing gap: related DeepMind work found that reinforcement learning fine-tuning (RLFT) raised action coverage from 40% to 52% and tic-tac-toe win rates from 15% to 75%—the gap is bridgeable, but requires new training methods.

Imitation vs. Understanding

Mimicry view: LLMs are statistical approximators optimized to generate artifacts, not adaptive responses; latent encoding of structure does not equal reliable task execution.
Understanding view: some capabilities mirror human cognition, and "artificial neuroscience" reveals functional similarities between models and brains.

As Blaise Agüera y Arcas and James Manyika write in AI Is Evolving — And Changing Our Understanding Of Intelligence: dramatic advances in AI are compelling us to rethink what intelligence truly is.

Paths Forward

1. Reinforcement learning fine-tuning (RLFT) — bridge reasoning and action through environmental reward. 2. Hybrid architectures — offload complex computation to external tools (calculators, map APIs, simulators). 3. Neurosymbolic AI — combine neural pattern recognition with symbolic logical reasoning. 4. Novel non-Transformer architectures that natively handle multidimensional structure.

References

1. Lepori, M. A., et al. (2025). *Language Models Struggle to Use Representations Learned In-Context*. https://arxiv.org/abs/2602.04212 2. Park, C. F., et al. (2025). *ICLR: In-Context Learning of Representations*. ICLR 2025. 3. DeepMind (2025). *LLMs Struggle to Act on What They Know*. https://www.marktechpost.com/2025/05/18/llms-struggle-to-act-on-what-they-know/ 4. Agüera y Arcas, B. & Manyika, J. (2025). *AI Is Evolving — And Changing Our Understanding Of Intelligence*. https://www.noemamag.com/ai-is-evolving-and-changing-our-understanding-of-intelligence 5. Subasioglu, M. & Subasioglu, N. (2025). *From Mimicry to True Intelligence (TI)*. arXiv:2509.14474.

True understanding lies not merely in knowing, but in being able to use what one knows. Until AI can walk the maps it draws, keep a clear-eyed respect for it—and double-check the route yourself when it matters.