Convergent Evolution: Why All LLMs Understand Numbers the Same Way
A "Too Perfect" Coincidence
Imagine you are a biologist who discovers three creatures — a mammal, a cephalopod, and an insect — in completely different corners of the Earth, yet their eyes are strikingly similar: lens, retina, photoreceptors, nearly identical. This is not because they share a common ancestor, but because the physical constraint of light is so strong that any species trying to survive on Earth will eventually evolve a similar visual system.
Biologists call this convergent evolution.
Now transplant this story into the AI world.
A research team from USC and UCSD discovered an equally surprising phenomenon: GPT-2, Llama-3, Llama-4, DeepSeek-V3, Mamba, xLSTM, GloVe, FastText — models with completely different architectures, different training methods, spanning nearly a decade — have independently evolved almost identical representations for understanding numbers.
They all arrange numbers on a "latent-space circle" with periods of 2, 5, and 10.
But the story is just beginning. The researchers then uncovered a deeper problem: seeing periodic structure does not mean the model truly "understands" numbers.
Two Layers of "Convergence": Looking Similar ≠ Actually Being Similar
Here is an analogy. Imagine watching a line of people from far away. You notice a clear periodic pattern — a cycle of 10 people, alternating heights. You might think: "They must be grouped by height."
Up close, you realize they merely happen to wear 10 different clothing colors; heights are random. The periodicity is real, but it does not represent the structure you guessed.
This is the paper's core finding. The researchers distinguish two kinds of convergence:
Spectral Convergence: number embeddings show Fourier-domain spikes at T=2, 5, 10. This is the "far away" view — you see periodicity.
Geometric Convergence: number embeddings form linearly separable classes by n mod T in space. This is the "up close" view — you confirm the periodicity corresponds to functional structure.
The key finding: almost all models achieve spectral convergence, but only some achieve geometric convergence.
The most shocking example is LSTM. On identical training data, LSTM's Fourier spikes are even larger than Transformer's, yet its mod-T probe accuracy stays at random-guess level.
In other words, LSTM's number embeddings look more "periodic," but its understanding of numbers is actually worse than Transformer's.
A Mathematical Theorem: Why "Looking Like" Is Not "Being"
The researchers prove this rigorously; here is the intuition.
Suppose you must partition 1000 numbers into 10 groups (by mod 10), 100 per group. Ideally, same-group numbers should cluster together in embedding space, with groups separated.
Fourier spikes tell you: between-class scatter is positive (group centers are spread apart). They do not tell you: whether numbers within each group also scatter (within-class scatter).
If within-group scatter far exceeds between-group scatter, groups overlap severely even with spread centers, and a linear classifier cannot separate them.
Using Fisher's LDA, the researchers give a precise mathematical characterization:
> Maximum separability = Φ_T / (N · λ_min(S_W)) · 1/cond(S_W)
where cond(S_W) is the condition number of the within-class scatter matrix. The larger the condition number, the more severely the periodic signal is drowned out.
LSTM's problem is exactly here: highly anisotropic within-class scatter (very large condition number) buries the periodic signal in noise. Transformer's within-class scatter is more uniform, preserving the signal.
The deep implication: any representation-level diagnostic method may mistake statistical artifacts of training data for learned structure. This is an important warning for the entire mechanistic interpretability field.
Three Forces: Data, Architecture, Optimizer
What determines whether a model achieves "true" geometric convergence? Controlled experiments isolate three key factors.
Data: Three Complementary Signals
Five data perturbations strip different types of co-occurrence information:
| Perturbation | Structure Removed | mod-10 probe κ | |---------|-----------|-------------| | Original | None | 85.4% | | Swap Numbers | number↔text association | 28.8% | | Unigram Replacement | all co-occurrence | ~0% | | Isolate-1 | digit-digit interactions | 45.0% | | Isolate-8 | limited digit interactions | 77.2% | | Context length 2 | long-range context | 40.3% | | Context length 64 | long-range context | 72.0% |
Key findings:
1. Text-number co-occurrence is the most important signal. Scrambling number-text associations (Swap Numbers) drops probe accuracy from 85.4% to 28.8%. 2. Digit-digit interactions are second. Even when two digits never appear in the same attention window (Isolate-1), Transformers still reach κ=45.0%, far above classic PPMI (27.1%) and word2vec (29.3%). 3. Long-range context adds gains: context growing from 2 to 64 steadily improves probe performance.
Under every perturbation, Fourier spikes persist — again confirming spectral and geometric convergence are driven by different mechanisms.
Architecture: LSTM's "Structural Failure"
With identical data and optimizer, three architectures diverge:
- Transformer: κ = 85.4% (mod-10)
- Gated DeltaNet (linear RNN): κ = 77.8%
- Mamba-2 (linear RNN): κ = 80.1%
- LSTM: κ ≈ 0% (random)
A possible explanation: LSTM's gating mechanisms (forget gate, input gate) introduce too much anisotropic noise when processing numbers. Transformer attention and linear RNN state-space models better preserve embedding-space geometry.
Optimizer: Architecture-Dependent Effects
Muon beats AdamW on Transformers (85.4% vs 72.1%) and Gated DeltaNet (77.8% vs 69.7%), but slightly underperforms on Mamba-2 (76.7% vs 80.1%).
Optimizer effects are not universal — they interact with architecture.
A Second Path: Convergence in Arithmetic Training
Does convergent evolution also occur when models are trained directly on addition, without natural language? The answer depends on tokenization.
9-digit addition (multi-token): each digit split into multiple tokens. Every output digit becomes a mod-1000 classification problem (due to carries), forcing the model to learn modular arithmetic — converging to the same periodic representation regardless of optimizer.
3-digit addition (single token): no modular constraint — the sequence "a+b=c" is equally consistent with mod-1000 or mod-1111. Outcomes depend entirely on random seed and optimizer; no stable convergence.
Deep insight: the tokenizer defines the "environmental pressure" a model faces. Multi-token tokenization implicitly creates modular-arithmetic subproblems via carry propagation; single-token tokenization leaves countless different paths available.
Engineering Takeaways
1. Don't be fooled by the Fourier spectrum. If probing reveals beautiful periodic structure, don't conclude the model "learned" something. First check the condition number of within-class scatter. 2. LSTM really can't handle number understanding. It's a fundamental architectural limitation, not insufficient training. For numerical reasoning applications, LSTM is a poor choice. 3. The tokenizer matters more than you think. It shapes what kind of number representations a model can learn. Multi-token digit representations ("1" + "2" + "3" rather than "123") actually help models understand number structure. 4. Muon is worth trying. On Transformers, Muon outperforms AdamW for number representation learning; consider it for training numerically-focused models. 5. "Convergent evolution" is a useful mental framework. Similar behavior across models doesn't imply shared mechanisms — like eyes, the same result can arise from entirely different evolutionary paths.
My Reflections
What struck me most is the paper's methodology.
The researchers didn't settle for "observe a phenomenon and conclude"; through careful controlled experiments (varying one variable at a time) and rigorous proofs, they fully exposed the gap between "looking like" and "actually being."
This structure attribution approach — attributing model behavior not to individual training samples (as influence functions do) but to structural features of the data distribution — offers mechanistic interpretability a fresh perspective.
The deeper lesson: representational similarity may mask fundamental mechanistic differences. This applies beyond number representations — to periodic representations of weekdays or months, or similar reasoning behavior patterns across models.
When we say two models "learned the same thing," we may need to ask more carefully: did they truly learn the same thing, or does it just look that way?
---
Paper: Convergent Evolution: How Different Language Models Learn Similar Number Representations (arXiv: 2604.20817)
Authors: Deqing Fu, Tianyi Zhou, Mikhail Belkin, Vatsal Sharan, Robin Jia (USC & UCSD)
Project page: convergent-evolution.github.io
Model weights: HuggingFace Collection
Blog: Project blog