Metaphor Has a Direction: Finding the Metaphor Switch in LLM Geometry

Forum topic · 小凯 · 2026-10-23

Summary

A University of Amsterdam study by Shihui Li, Mette Andersen, and Martha Lewis shows that metaphors have a causally manipulable geometric direction inside large language models. Using activation patching and Distributed Alignment Search (DAS), the researchers identified low-dimensional subspaces that causally control whether a model judges text as metaphorical or literal. Experiments covered three word classes (nouns, adjectives, and domain-specific adjectives) and three tasks (identification, interpretation, and generation) across multiple LLM architectures, including Llama 3. Intervening along the discovered subspace increased or decreased the metaphorical quality of generated text. The findings suggest LLMs encode conceptual mappings underlying metaphor rather than mere rhetorical patterns, echoing earlier discoveries of 'truthfulness' and 'refusal' directions. Limitations include testing mainly on small-to-mid-scale models, unexplored metaphor subtypes, and no released code.

"Time is money" — what does this look like inside an LLM?

"Time is money," "argument is war," "love is a journey." These metaphors are not just rhetorical decoration. Lakoff and Johnson's 1980 classic argued that metaphor is a fundamental mechanism of human cognition: we understand one domain through another.

If metaphor is truly cognitive infrastructure, then large language models should contain some internal structure representing it. What Shihui Li, Mette Andersen, and Martha Lewis at the University of Amsterdam did was to find the "direction" of metaphor in the geometric space of LLMs — and then control it like a knob.

Method: Finding the "metaphor subspace" in the residual stream

The paper uses two mechanistic interpretability techniques:

1. Activation Patching

The idea is straightforward: if an activation vector at some position encodes "is this a metaphor," then replacing it with a different activation should change the model's metaphor judgment.

Concretely: while the model processes "time is money," patch an activation from processing "time is a resource" and see whether the output flips from literal to metaphorical.

2. Distributed Alignment Search (DAS)

DAS goes further: instead of directly swapping activations, it finds a low-dimensional subspace such that interventions within that subspace causally change the model's metaphor judgment.

It's like finding a "metaphor axis" in a high-dimensional space — push along it and output becomes more metaphorical; push the other way and it becomes more literal.

The paper compares DAS with the mass-mean steering method of Marks & Tegmark (2024), which computes the difference between mean activations of metaphorical vs. non-metaphorical samples. DAS is more refined: it finds causally effective directions rather than statistical averages.

Three datasets, three tasks

Experiments cover three word classes:

NOUN: noun metaphors ("time is money")
ADJ: adjective metaphors ("a cold gaze")
DOM: domain-specific adjectives (metaphorical usage in specific contexts)

Three tasks:

Identification: is this sentence metaphorical or literal?
Interpretation: what does this metaphor mean?
Generation: have the model produce metaphorical or literal text

Results: metaphor has a direction, and it can be controlled

Core findings:

On classification tasks: across multiple LLMs (different architectures, different scales), causally effective subspaces can be found such that intervening along them changes the model's metaphor judgments. This holds across word classes — nouns, adjectives, and domain-specific adjectives.

On generation tasks: intervening in this subspace controls the "metaphorical degree" of generated text. Turn it up for more metaphorical output; turn it down for more literal output.

This means: metaphor in LLMs is not some scattered, unlocalizable phenomenon — it has geometric structure, and it is localizable and manipulable.

Why this matters

1. Metaphor is a window into cognition

If LLMs have a "metaphor direction" internally, it suggests the model learned the conceptual mappings behind metaphor — understanding one domain structurally through another — not just metaphor as a rhetorical device.

2. Possibilities for controllable generation

Controlling metaphorical degree means you can:

Lower metaphoricity when precision matters (medical, legal settings)
Raise it for creative writing (literature, advertising)
Study how different models and training stages encode metaphor

3. Another proof point for mechanistic interpretability

This paper is a concrete case of the broader direction of "finding causal structure inside models." Previous work found "truthfulness directions," "emotion directions," and "refusal directions." Now there's a "metaphor direction" — more and more high-level semantic concepts are found to have corresponding geometric structure.

The picture is getting clearer: LLM internals are not a black box but a structured geometric space where different semantic dimensions correspond to different directions. We are learning to read this map.

An honest assessment

Limitations of the paper:

Experiments are mostly on small-to-mid-scale models (e.g., Llama 3); effects at much larger scale are unverified
"Metaphor" is a fuzzy concept — different types (dead metaphors, extended metaphors, mixed metaphors) may correspond to different internal structures, and the paper doesn't distinguish them
Whether the DAS-found subspaces "align" across models (same direction = same meaning) is not discussed
Generation quality was evaluated with LLM-as-judge, which has known reliability issues
No code released

But as a proof of concept — demonstrating that metaphor has a localizable geometric structure in LLMs — the paper delivers what it set out to do.

---

Paper: arxiv.org/abs/2610.31520