An Intuitive Question
Show a multimodal model two problems:
- Text input: "17 + 35 ="
- Image input: a white image with "91 - 39" written in black text
- Direct reuse: 53.2% recovery
- Rotation \(Q\) only: 71.8% recovery
- Rotation + scaling \(\alpha Q\): 91.4% recovery
- Unconstrained linear map: 96.3% (only 5 points more)
- Text → image: \(\alpha \approx 0.65\) (visual representations are compressed)
- Image → text: \(\alpha \approx 1.28\) (viewed the other way, text representations are ~1.5× "larger" than visual ones)
- D (9-dim): coupled to the readout, directly controlling which digit is the next token
- L (13-dim): lies in the null space of \(U_{dig}\) — invisible to the output layer
- Cross-condition displacement cosine similarity: 0.807
- Top-1 retrieval accuracy on unseen values: 75.5% (random baseline: 5%)
- Top-5 retrieval: 98.8%
- Fit-free RSA check: \(\rho = 0.765\) (random 13-dim subspace: 0.545)
- Independently fit pairwise maps: 73.4% top-1 retrieval
- Global synchronization: 74.8% top-1 retrieval (essentially unchanged)
- Holding out one pair of conditions and reconstructing it from the other ten: 71.1% top-1
- The L subspace reliably encodes complete numbers (75.5% retrieval)
- Yet intervening on L alone changes the model's output in only 3.2%–6.4% of cases
- First digit (tens): unaffected 96.5% of the time (D controls it)
- Second digit (units): follows L 77.6% of the time (the units digit 7 from ã), not D
- Experiments cover only addition/subtraction in [0, 99]. Multiplication representations are more dispersed and depth-dependent, and were not included in the four-condition analysis
- Only two models tested: Gemma4-12B-it and Ministral-3-14B
- Code "coming soon" (as of paper writing)
- No localization of which attention heads perform the L→D transformation
- Why chain-of-thought works: perhaps not because the model "thinks step by step," but because each generation step gives L's precomputed results an opportunity to be released into D
- Why models sometimes "know but say wrong": perhaps the answer in L is correct, but the L→D transformation fails at some step
- Why multimodal alignment is hard: not just "image and text spaces differ," but "they share geometry in different coordinate systems, and the visual coordinate system's ruler is shorter"
Both answers are 52, and the model outputs the same number in both cases. So the question is: is the "52" in the model's head one concept, or do two independent computation pipelines each arrive at the same result?
This is much harder to answer than it looks. Traditional representation-similarity methods (RSA, CKA) will mislead you here — they give "exactly identical directions" and "rotated directions" the same score, and they can produce a false sense of similarity just because both representations are preparing to output the same digit.
A research team from Apple and UAB used causal interventions to uncover a surprising structure in the residual stream of Gemma4-12B-it: four conditions (text addition, text subtraction, image addition, image subtraction) share the same numerical geometry, but express it in different coordinate systems. Even more surprisingly, most of this shared geometry hides in a place the model's output layer cannot "see."
Paper: Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation
Why Similarity Methods Are Not Enough
Suppose you want to confirm whether "the 52 in text" and "the 52 in an image" are the same thing inside the model. The most intuitive approach: extract the residual-stream activations for both cases and compute cosine similarity or CKA. If similarity is high, conclude they are "the same."
The problem: high similarity does not mean the same causal mechanism.
Imagine two coordinate systems — one using latitude/longitude, another using polar coordinates — for the same location. The coordinate vectors look completely different but describe the same point. Conversely, two vectors can look similar merely because both are preparing to output the same digit token — unrelated to whether a numerical concept is shared.
The team's choice: don't look at what representations look like; look at what happens when you change them.
Causal Intervention: DAS Subspace Swapping
The method is called Distributed Alignment Search (DAS), and its core idea is straightforward:
1. In the layer-43 residual stream of Gemma4-12B-it, find a 22-dimensional subspace 2. Train an orthogonal basis R such that swapping coordinates between two different inputs inside the subspace swaps the model's answers 3. If the swap succeeds, this subspace causally controls the model's representation of numbers
Each of the four conditions gets its own subspace: text addition \(R_{T+}\), text subtraction \(R_{T-}\), image addition \(R_{I+}\), image subtraction \(R_{I-}\).
Analogy: imagine the model has four offices, each with a safe storing numbers. DAS finds the safe's coordinates in each office and verifies that "moving the contents from office A's safe to office B's safe" changes office B's output accordingly.
Key Data
Autoregressive intervention accuracy (IIA_AR) for the 22-dimensional subspace across the four conditions:
| Condition | IIA_AR | |------|--------| | Text addition | 69.3% – 86.2% | | Text subtraction | same range | | Image addition | same range | | Image subtraction | same range |
Meanwhile, using only Fourier-aligned directions (very effective for Llama in prior work) recovers only 23% of the causal effect in Gemma4 — far below Llama3.1-8B's 84%. Numerical representations in modern multimodal models are far more distributed than in earlier LLMs.
Finding 1: Different Directions, Same Geometry
Applying the text-addition subspace \(R_{T+}\) directly to image addition recovers only 53.2% of the causal effect; cross-modal direct reuse averages 45.5%.
Conclusion 1 (answers Q1 negatively): the four conditions do not encode numbers along the same residual-stream directions.
But the story doesn't end there. The team fit a scaled rotation \(\alpha Q\) mapping the source condition's subspace displacement to the target condition:
One rotation plus one scalar scaling achieves nearly everything an unconstrained linear map can.
Even more interesting is the physical meaning of the scaling factor \(\alpha\):
Analogy: the numbers of all four conditions live in the same building, but each office's coordinate system is rotated differently, and the visual office's ruler is shorter than the text office's. Once you know the rotation angles and the ruler ratios, you can translate numbers between any two offices.
Compositionality
An even nicer experiment: when changing both modality and operation simultaneously (e.g., text addition → image subtraction), a directly fit map recovers 80.7%. But applying a modality rotation followed by an operation rotation (two steps) recovers 79.9% — nearly identical.
This means modality and operation geometric transformations are composable, like multiplying two independent rotation matrices.
Finding 2: The Numerical Geometry Is Not in the Output Layer
A reasonable suspicion at this point: perhaps the four conditions share geometry only because they are all preparing to output the same digit token. What is shared is not a "number concept" but the "output interface."
The team performed a precise decomposition:
1. Take the unembedding vectors \(W_{dig}\) for digits 0–9 2. Subtract the mean and orthogonalize to get a 9-dimensional "readout basis" \(U_{dig}\) 3. Sort the 22-dimensional causal subspace by alignment with \(U_{dig}\) and cut at dimension 9 4. This yields two orthogonal subspaces:
If D is projected away, autoregressive intervention accuracy plummets from 82.6% to 0.8%. D carries nearly all immediate output control.
But what does L contain?
L contains complete numbers.
Within the L subspace:
Analogy: imagine the model is a student taking an exam. D is the digit he is currently writing; L is the rest of the answer he has already computed but hasn't written down yet. The teacher (the output layer) can only see D, not L. But L genuinely stores the answer.
Global Synchronization
A further experiment: can the L subspaces of the four conditions be synchronized into a unified coordinate system?
Using orthogonal group synchronization to align the four conditions' 13-dimensional coordinate systems simultaneously reduces parameters from 948 to 237. Results:
The four conditions share a single 13-dimensional numerical coordinate system, each rotated differently.
Forcing addition and subtraction within the same modality to share a coordinate system drops retrieval accuracy from 74.8% to 18.1% — different operations genuinely require different coordinate systems.
Finding 3: L Controls the Next Digit
Here an apparent contradiction appears:
L has information but seems not to affect output. Why?
The answer lies in autoregressive generation: the model must output the answer digit by digit, but arithmetic computes the full result at once.
The team designed an elegant "mismatched intervention" experiment:
1. Take a base input (e.g., 17+14=31) 2. Take two donors: y=52 and ã=57 (sharing the tens digit 5, differing in units digit) 3. Construct a hybrid state: D from y=52, L from ã=57 4. See what the model outputs
Results:
D controls the current token; L controls the next token.
More precise tracing: at layer 44 after intervention, L's modification has near-zero effect on readout directions (\(\gamma_{44} \approx 0.11\)). But at layer 45 — once attention reads the modified KV-cache — the effect jumps to \(\gamma_{45} \approx 0.72\), reaching a median of 0.85 at the final layer.
L uses attention at the next step to translate itself into directions D can understand.
Analogy: imagine a simultaneous interpreter translating a long number. When he hears "fifty-two," his mouth says "5" (D at work), but his mind has already stored "2" ready for the next utterance (L buffering). L doesn't "fail to affect output" — it affects the output of the *next* step.
Why This Matters
1. "Similar Representations" ≠ "Same Mechanism"
This paper adds an important qualification to the Platonic Representation Hypothesis (that representations across modalities converge to a shared model of reality): what converges is geometric structure, not coordinate directions. If you only look at RSA/CKA, you overestimate alignment because you conflate "the same structure, rotated" with "truly identical directions."
2. Computation and Output Are Decoupled
The model does not "compute one step, output one step." It computes the complete answer at layer 43, stores it in L, then releases it digit by digit into D via attention. This aligns with Anthropic's "latent planning" finding in Claude — the model has "thought through" the full answer in its internal representation before outputting it.
3. Visual Representations Are Compressed
\(\alpha \approx 0.65\) is no accident. Numerical displacements in the visual modality are systematically smaller than in text. This may explain why cross-modal direct subspace reuse works better image→text than text→image — visual representations are "smaller," so expanding them into text space is easier, while compressing loses information.
4. Fourier Is Not the End of the Story
On Llama3.1, Fourier directions recover 84% of the causal effect. On Gemma4, only 23%. As models get stronger and more multimodal, numerical representations evolve from "neat Fourier periodic structures" to "more dispersed causal subspaces." Better models don't necessarily have tidier internal structure — possibly the opposite.
Limitations of the Method
Engineering Takeaways
If you work on multimodal interpretability:
1. Don't rely on representation similarity alone. RSA and CKA will mislead you. Causal intervention is the only reliable way to verify genuine sharing. 2. The output layer doesn't see everything. Models contain substantial structure in the null space of the readout. If you only analyze directions near the unembedding, you'll miss buffers like L. 3. Cross-modal alignment must account for scaling. Not just rotation, but representational magnitude differences. Visual representations may be more "compact" than text. 4. Timing matters. In autoregressive models, the same subspace plays different roles at different time steps. L is "hidden" at the current step and "controlling" at the next.
Personal Reflections
What fascinates me most about this paper is the "mismatched intervention" experiment. Point D at 52 and L at 57, and the model writes 5 then 7. This isn't "the model being fooled" — it's "the model working according to its own temporal plan": it computed the full answer long ago and is just waiting for attention to move it from L to D.
This may connect deeply with many phenomena we see in LLMs:
---
Paper: arXiv:2610.37422 Authors: Oriol Juan Sabater (UAB), Alex Ferrando de las Morenas, Pau Rodríguez, Xavier Suau (Apple) Models: Gemma4-12B-it, Ministral-3-14B Code: "coming soon" per the paper