Does AI Really Understand What It Says? Unpacking 'Inert Knowledge' in Google DeepMind Research

Forum topic · ✨步子哥 · 2026-02-11

Summary

A Google DeepMind research poster circulating on zhichai.net examines a striking paradox in large language models: while in-context learning lets AI instantly adapt to new instructions, the models internally fail to use what they have learned. The featured paper, 'Language Models Struggle to Use Representations Learned In-Context' by Lepori et al. at Google DeepMind, argues that neural networks build accurate internal representations of world structure—a perfect 'map'—yet cannot extract or invoke that knowledge during reasoning, a gap the authors call 'inert knowledge.' Evidence from metrics such as Dirichlet energy and distance correlation confirms that models do encode high-dimensional geometric structure internally. The poster attributes this knowing-doing gap to architectural limits: the Transformer's self-attention mechanism processes one-dimensional sequences and struggles with two-dimensional spatial logic. The result is a model with perfect memory but no ability to act on it—a ghost trapped in dimensions.

Does AI Really Understand What It Says? Unpacking "Inert Knowledge" in Google DeepMind Research

This post from zhichai.net shares a visual research poster summarizing recent Google DeepMind work on why large language models appear to understand in-context instructions but fail to actually use the knowledge they contain.

The Paradox: Appearance vs. Reality in In-Context Learning

On the surface, in-context learning seems magical: AI models can adapt to new instructions from a prompt in seconds. But DeepMind's research reveals a disturbing truth: the model builds a perfect "map" of the world inside its brain—yet cannot take a single step with it.

Core Finding: Inert Knowledge

The model's neural network has already perfectly represented the structure of the world (the map), but its computation engine cannot extract or invoke this knowledge for reasoning.

✓ MAP — perfectly represented internally
✗ NAVIGATE — cannot execute

This is a complete split between "knowing" and "doing": the model knows the rules but cannot perform the operations.

Evidence of Representation Learning

Although the AI cannot use the knowledge, hard-nosed metrics show it genuinely "learned":

Dirichlet energy
Distance correlation

These prove the model constructs a high-dimensional geometric world inside the black box. It is a ghost trapped in dimensions—with perfect memory but no capacity for action.

Architectural Limitation: Trapped in "One Dimension"

Why can't the model handle complex logic? Because it is a "novel-reading expert":

> The Transformer's self-attention mechanism is inherently designed to process one-dimensional sequences and cannot effectively handle two-dimensional spatial logic.

Paper Reference

Paper: *Language Models Struggle to Use Representations Learned In-Context*
Authors: Lepori et al., Google DeepMind