Does AI Really Understand What It Says? Google DeepMind Research Unveils the Truth About "Inert Knowledge"

Forum topic · ✨步子哥 · 2026-02-11

Summary

A Chinese tech forum post presents an infographic-style explainer of Google DeepMind research (Lepori et al., "Language Models Struggle to Use Representations Learned In-Context") on the gap between what AI models know internally and what they can actually use. The key concept is "inert knowledge": neural networks may form accurate internal representations of world structure — a perfect "map" — yet fail to extract or apply this knowledge during reasoning and execution. The post cites evidence from representational-learning metrics, including Dirichlet energy and distance correlation, showing that models do build high-dimensional geometric representations inside the black box. It also highlights an architectural limitation: Transformer self-attention processes one-dimensional sequences and struggles with two-dimensional spatial logic. The paradox framed is in-context learning — models appear to instantly grasp new instructions from prompts, yet cannot navigate the knowledge they encode, a fundamental split between knowing and doing.

Does AI Really Understand What It Says? Unveiling the Truth About "Inert Knowledge"

This forum post is an infographic-style explainer (original in HTML/CSS) summarizing a Google DeepMind research paper.

Paper: *"Language Models Struggle to Use Representations Learned In-Context"* — Authors: Lepori et al., Google DeepMind.

Appearance vs. Reality: The In-Context Learning Paradox

On the surface, In-Context Learning makes AI appear to instantly understand new instructions from prompts, almost like magic. But DeepMind's research reveals a disturbing truth: the AI has built a perfect "map" in its brain — yet cannot take a single step.

Core Finding: Inert Knowledge

The neural network has internally represented the structure of the world (the map) perfectly, but its computation engine cannot extract or invoke this knowledge for reasoning.

✓ MAP — perfectly represented
✗ NAVIGATE — cannot execute

This is a complete split between "knowing" and "doing". The model knows the rules but cannot perform the operations.

Evidence of Representation Learning

Although the AI cannot *use* its knowledge, hard metrics prove it has indeed "learned":

Dirichlet energy
Distance correlation

These demonstrate that the AI has genuinely constructed a high-dimensional geometric world inside the black box. It is a ghost trapped in dimensions — perfect memory, no agency.

Architectural Limitation: Trapped in "One Dimension"

Why can't AI handle complex logic? Because it is a "novel-reading expert":

> The Transformer's Self-Attention mechanism is inherently designed to process one-dimensional sequences and cannot effectively handle two-dimensional spatial logic.

Takeaway

Large language models may encode correct world models but fail to leverage them at inference time — a gap the DeepMind paper quantifies, suggesting future architectures may need mechanisms beyond 1D attention to bridge knowing and doing.

*Original post images: brain AI visualization, Dirichlet energy visualization, self-attention heatmap (hosted on sfile.chatglm.cn).*