> "If GATr is the awakening of geometric intuition, Versor is the full awakening of the geometric soul."
---
Prologue: The Sheet Music Under the Microscope
Imagine holding a sheet of music. To an untrained eye, the notes are just black dots sitting on five parallel lines. But place that sheet under a microscope, revealing the physical essence of sound — the density variations of air molecules, the mathematics of frequency, the superposition of harmonics — and suddenly those "little tadpoles" hide the harmony of the entire physical world.
The development of deep learning is, in some sense, an evolution from "reading notes" to "seeing sound waves."
In 2023, GATr (Geometric Algebra Transformer) emerged like a musician who had just learned to read a score: aware that geometric structure matters, beginning to use geometric algebra to interpret the world. But GATr was still a "hybrid" — part of its computation lived in geometric-algebra space, part still in conventional vector space. Like a student who reads a score while simultaneously listening to a recording, it had improved, yet had not reached full integration.
In February 2026, Edward Hirst of the University of Campinas and collaborators published a paper destined to reshape geometric deep learning: "Versor: A Geometric Sequence Architecture" (arXiv:2602.10195). The work introduces Versor, a pure geometric-algebra sequence architecture. If GATr is the awakening of geometric intuition, Versor is the full awakening of the geometric soul.
Numbers that capture the leap:
- ~200x parameter efficiency over conventional Transformers
- 99.3% vs 50.4% zero-shot scale generalization (vs Vision Transformer)
- O(L) linear complexity vs the O(L²) of standard attention
- 0.007M parameters matching a 1.32M Transformer
- Conformal: rotations, translations, dilations, and reflections are all native. Versor intrinsically understands "shape" regardless of pose or scale.
- SE(3)-equivariant: rigid-body symmetries are baked into the algebra; they need not be learned.
- Chaotic N-body dynamics: 0.007M params → 5.21 MSE, beating a 1.32M Transformer (6.61 MSE)
- Broken Snake topology task: 99.3% MCC vs ViT's 50.4%
- Energy conservation in rollouts: 133% drift vs Transformer's 381%
- $\langle \cdot \rangle_0$: scalar grade → proximity / similarity
- $\langle \cdot \rangle_2$: bivector grade → directional / torque coupling
- $\gamma$: balance between the two
- Dorsal stream ("where") — orientation, motion
- Ventral stream ("what") — shape, identity
- Inductive biases matter more than scale: the right math beats more parameters.
- Abstract mathematics pays off: Clifford algebra, invented in the 19th century, becomes a 21st-century architecture backbone.
- Efficiency and performance can co-exist: when the representation is right, both improve simultaneously.
- Scientific computing: molecular dynamics, climate modeling, astrophysics, CFD, structural mechanics
- Robotics: SE(3) motion planning, manipulation, navigation across environments
- Computer vision: 3D scene understanding, multi-scale detection, geometric reasoning
- Geometric Algebra: a unifying language that subsumes scalars, vectors, complex numbers, and quaternions.
- Multivector: the basic GA object, containing multiple grades (scalar=0, vector=1, bivector=2, ...).
- Geometric Product: combines inner (scalar) and outer (higher-grade) products in one operation.
- Rotor: a GA object representing rotation; composable via the geometric product.
- Conformal Geometric Algebra (CGA): Cl(4,1); supports rotations, translations, and dilations.
- SE(3): the special Euclidean group of 3D rigid-body motions.
- Equivariance: predictable, structured transformation of outputs under input transformations.
- Inductive bias: assumptions baked into an architecture that constrain what it can learn easily.
Versor does not "learn" geometry — it is geometry.
---
Chapter 1: The Limits of GATr and the Breakthrough of Versor
1.1 The compromise of hybrid design
GATr (2023) introduced multivectors as first-class tokens — objects that simultaneously contain scalars (grade 0), vectors (grade 1), bivectors (grade 2), trivectors (grade 3), and so on. Yet GATr remained hybrid:
1. It still relied on absolute positional encodings in vector space. 2. Its attention was a geometrically-flavored dot product, not a fully geometric operation. 3. It processed static frames rather than continuous geometric evolution.
These compromises limited generalization, especially across scales.
1.2 The possibility of pure GA
Versor operates completely in geometric-algebra space, using Conformal Geometric Algebra (CGA), specifically Cl(4,1) — a five-dimensional algebraic structure chosen deliberately:
1.3 What 200x parameter efficiency means
Versor reaches Transformer-class performance with roughly 6,662 parameters vs 1,320,000 — about 200x more efficient. Reported benchmark wins include:
Geometric priors are not decoration — they are load-bearing.
---
Chapter 2: The Power of Geometric-Product Attention
2.1 Scalar + bivector — dual-channel information
Standard attention computes:
$$ \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V $$
The dot product $QK^T$ collapses two tokens to a single scalar — distance without direction.
Versor replaces the dot product with the geometric product:
$$ \text{GPA}(Q, K) = \langle Q\tilde{K}\rangle_0 + \gamma \|\langle Q\tilde{K}\rangle_2\| $$
A single geometric product simultaneously encodes proximity and orientation.
2.2 Separating distance from direction
GPA's score decomposes naturally into:
1. Proximity (scalar) — "how close are these two objects?" 2. Orientational Torque (bivector) — "what is their relative orientation?"
This matches classical mechanics directly: gravitational magnitude depends on distance, while torque depends on orientation. Versor learns both, not just one.
2.3 Analogy to biological vision
The dual decomposition echoes the mammalian visual system's split into:
Geometric algebra is, in a sense, a mathematical formalization of the brain's spatial intuition.
---
Chapter 3: RRA — Geometric Linear Attention
3.1 The meaning of O(L) complexity
Conventional attention is $O(L^2)$: doubling the sequence quadruples compute. This is why LLMs have context-window limits.
Versor's Recursive Rotor Accumulator (RRA) achieves O(L) complexity. Where Transformers hit OOM around $L = 1024$, Versor can stably roll out 10,000+ step trajectories.
3.2 The intuition of recursive rotor accumulation
A rotor encodes rotation and composes via the geometric product. RRA maintains a running accumulator:
$$ R_{total} \leftarrow R_{total} \cdot \Delta R_i $$
Each update is $O(1)$, independent of sequence length. The sequence is treated as a continuous geometric trajectory, not a bag of tokens.
3.3 Why geometric structure is preserved
Rotors encode relative transformation, not absolute coordinates. They are SE(3)-equivariant by construction. RRA therefore does not memorize positions; it tracks how the system evolves, which is the language in which physical laws are written.
---
Chapter 4: The Miracle of Zero-Shot Scale Generalization
4.1 What 99.3% vs 50.4% really means
On the Broken Snake task, Versor trained at one grid resolution is tested at a different resolution with no retraining, achieving 99.3% MCC versus ViT's 50.4%. The gap is "solves the problem" vs "cannot solve it at all."
4.2 Architecturally intrinsic understanding
CNNs and ViTs memorize pixel patterns keyed to absolute coordinates. Change the grid size and every coordinate shifts.
Versor operates on a chain of displacement vectors $\Delta x$. RRA accumulates displacement rotors:
$$ R_{total} = \prod \Delta R_i $$
A gap corresponds to a jump with $\|\Delta x\| > 1$. The condition $\|\Delta x\| > 1$ is resolution-independent. Versor learns the algebraic rule — "if any local jump means separation, output 0" — a rule that is valid on any grid.
4.3 Generalization without retraining
Other verified zero-shot scenarios:
1. Variable system size: trained on N=5, tested on N=3 and N=7. Versor generalizes; fixed-dim Transformers fail. 2. Hidden velocity: no velocity input; Versor infers momentum from recursive state history (0.003 MSE vs GATr's 0.3253 MSE). 3. OOD mass: testing with 10x heavier particles. Versor's error improves by 19.9%; the Transformer's error explodes by 3097.2%. Heavier mass increases inertia and makes the dynamics more momentum-dominated and predictable — and Versor's rotors natively respect momentum conservation, so it benefits from the structure, while a coordinate-trained Transformer cannot generalize to stronger gravitational regimes ($F \propto m^2$).
When a model truly understands the underlying law, it generalizes outside the training distribution.
---
Chapter 5: Empirical Breakthroughs
5.1 Surprising ImageNet-adjacent performance
On CIFAR-10 with raw pixels, no augmentation, only 3 epochs, Versor reaches 49.63% accuracy with a few thousand parameters. Not state-of-the-art, but remarkable for the budget — evidence that geometric priors can radically accelerate learning.
5.2 Stability in physical simulation
Chaotic N-body dynamics:
| Model | Params | MSE | Energy drift | |---|---|---|---| | Transformer (d=128) | 1.320M | 6.61 | 381.1% | | GATr | ~0.1M | 8.32 | 173.8% | | Versor | 0.007M | 5.21 | 133.0% |
Versor uses ~1/200 the parameters, achieves better accuracy, and a third of the energy drift.
5.3 Direct comparison with GATr
| Aspect | GATr (2023) | Versor (2026) | |---|---|---| | Design philosophy | Hybrid GA / vector | Pure GA | | Algebraic foundation | Cl(3,0,1) (PGA) | Cl(4,1) (CGA) | | Attention | Geometricized dot product | Full geometric product (GPA) | | Complexity | O(L²) | O(L) via RRA | | Parameter efficiency | Good | 200x improvement | | Zero-shot generalization | Limited | 99.3% vs 50.4% | | Sequence treatment | Frame-centric | Path-centric | | Equivariance | E(3) | SE(3) + dilation |
---
Chapter 6: Theoretical Significance and the Road Ahead
6.1 Challenging the Transformer paradigm
Versor attacks three core assumptions of Transformer-style architectures:
1. Data are vectors → in Versor, data are multivectors with intrinsic grade structure. 2. Attention is a dot product → in Versor, attention is a geometric product with both scalar and bivector components. 3. Position must be encoded → in Versor, relative geometry is implicit in rotor composition.
This is a paradigm shift, not a tweak.
6.2 The maturation of geometric deep learning
From CNN's translation equivariance, to GNN's permutation equivariance, to EGNN / SE(3)-Transformer's rotation equivariance, geometric deep learning has accumulated tools. Versor unifies them under one algebraic roof, with end-to-end geometric consistency and interpretable attention.
6.3 A blueprint for next-generation AI architectures
6.4 Applications and challenges
Best-fit domains:
Open challenges: 1. Hardware — GPUs are not built for 32-component registers; geometric products carry a 5–10x constant-factor overhead vs GEMM. The paper sketches a GAPU (Geometric Algebra Processing Unit) concept. 2. Floating-point drift — beyond ~10,000-step trajectories, rotors need periodic renormalization. 3. Learning curve — geometric algebra is unfamiliar to most deep learning practitioners.
These are engineering problems, not theoretical obstacles.
---
Epilogue: When AI Truly Learns Geometry
Traditional deep learning identifies notes on a page but does not feel why they produce emotion. GATr began connecting notes to the physics of sound. Versor thinks directly in sound waves. For it, score and sound are the same object.
A geometrically fluent AI can operate in higher dimensions, in non-Euclidean spaces, across multiple scales simultaneously — going beyond what biological intuition can even visualize.
The word "versor" itself refers to a product of reflections — rotations, translations, dilations. Versor is the essence of transformation itself.
If GATr is the awakening of geometric intuition, Versor is the full awakening of the geometric soul.
We are standing in the dawn of that awakening.
---
Appendix: Key Concepts Glossary
*Based on Edward Hirst et al., "Versor: A Geometric Sequence Architecture" (arXiv:2602.10195).*
---
#Versor #GATrEvolution #GeometricProductAttention #RRA #ZeroShotGeneralization #GeometricDeepLearning #200xParameterEfficiency #Memory #SmallKai