Versor: When AI Truly Learns Geometry — The Evolution Beyond GATr and the Dawn of Geometric Deep Learning

> "If GATr was the awakening of geometric intuition, Versor is the full awakening of the geometric soul."

This post analyzes the paper *Versor: A Geometric Sequence Architecture* (arXiv:2602.10195), published in February 2026 by Edward Hirst and colleagues at the University of Campinas, Brazil. Versor is a pure geometric algebra sequence architecture that reportedly achieves:

• 200x parameter efficiency improvement over a traditional Transformer
• 99.3% vs 50.4% zero-shot scale generalization accuracy vs a Vision Transformer
• O(L) linear complexity vs the O(L²) of standard attention
• 0.007M parameters matching the performance of a 1.32M-parameter Transformer

Key points

1. From GATr's hybrid compromise to pure geometric algebra

• GATr (2023, "Geometric Algebra Transformer") replaced plain vectors with multivectors (scalars, vectors, bivectors, trivectors), but remained a hybrid design: it still used absolute position encoding, dot-product-style attention, and frame-centric sequence processing.
• Versor operates entirely in conformal geometric algebra (CGA), specifically \(Cl_{4,1}\). Conformal transformations (rotation, translation, scaling, reflection) preserve angles, and SE(3) equivariance is built into the algebra rather than learned.
• The headline parameter result: Versor reaches Transformer-level performance with 6,662 parameters vs 1,320,000 — a 200x reduction — because it learns geometric structure rather than memorizing coordinate-specific patterns.

2. Geometric Product Attention (GPA)

Standard attention relies on the dot product \(QK^T\), which keeps only distance/similarity information and discards direction. GPA replaces it with the full geometric product:

\[GPA(Q, K) = \langle Q\tilde{K} \rangle_0 + \gamma\|\langle Q\tilde{K} \rangle_2\|\]
• The grade-0 scalar part encodes proximity (traditional attention-like similarity).
• The grade-2 bivector part encodes directional coupling — an "orientational torque" analogous to how gravity produces torque depending on relative orientation, not just distance.
• This decomposition makes attention interpretable and is compared to the brain's dorsal ("where") vs ventral ("what") visual streams.

3. Recursive Rotor Accumulator (RRA): linear-time memory

• Instead of treating a sequence as independent tokens, RRA treats it as a continuous trajectory of rotations, maintaining a cumulative rotor: \(R_{total} \leftarrow R_{total} \cdot \Delta R_i\) per step — an O(1) update, yielding O(L) overall complexity.
• Reported capability: Transformers exhaust memory around L=1024, while Versor handles trajectories of 10,000+ steps.
• Because rotors encode relative geometric evolution, the representation is SE(3)-equivariant and does not degrade with changes of coordinate frame.

4. Zero-shot scale generalization

On the Broken Snake task (detecting whether a snake-like path on a grid is continuous or broken by a one-pixel gap):

• Trained at one grid resolution, tested at another: Versor achieves 99.3% MCC vs 50.4% for a Vision Transformer.
• Versor consumes chains of displacement vectors \(\Delta x\) rather than absolute coordinates; the rule \(\|\Delta x\| > 1\) indicates a gap and is resolution-independent.
• Additional zero-shot/OOD results cited:
• Variable system size: trained on N=5 particles, generalizes to N=3 and N=7; fixed-dimension Transformers fail entirely.
• Hidden velocity: infers momentum from recursive state history (0.003 MSE) where frame-based GATr fails (0.3253 MSE).
• OOD mass: with test masses 10x heavier, Versor's error improves by -19.9% (heavier masses make trajectories more momentum-dominated), while a Transformer degrades by +3097.2%.

5. Benchmark results

Chaotic N-body dynamics:

| Model | Params | MSE | Energy drift | |------|--------|-----|----------| | Transformer (d=128) | 1.320M | 6.61 | 381.1% | | GATr | ~0.1M | 8.32 | 173.8% | | Versor | 0.007M | 5.21 | 133.0% |

CIFAR-10: using raw pixels with no augmentation, Versor reaches 49.63% accuracy in 3 epochs — suggesting geometric inductive bias dramatically accelerates early learning.

6. GATr vs Versor

| Feature | GATr (2023) | Versor (2026) | |------|-------------|---------------| | Design philosophy | Hybrid GA/vector | Pure GA | | Algebra | \(Cl_{3,0,1}\) (PGA) | \(Cl_{4,1}\) (CGA) | | Attention | Geometrized dot product | Full geometric product (GPA) | | Complexity | \(O(L^2)\) | \(O(L)\) via RRA | | Parameter efficiency | Good | 200x | | Zero-shot generalization | Moderate | 99.3% vs 50.4% | | Sequence processing | Frame-centric | Path-centric | | Equivariance | E(3) | SE(3) + scaling |

7. Significance, applications, and open challenges

• The post argues Versor challenges core Transformer assumptions: data as plain vectors, attention as dot products, and position as something that must be explicitly encoded. Geometry, here, is not a learned feature but the substrate of the architecture itself.
• Suitability: physics simulation (molecular dynamics, climate, astrophysics), robotics (motion planning, manipulation, scale-invariant navigation), and 3D vision / multi-scale detection.
• Remaining challenges are engineering, not theoretical: current GPUs are not optimized for the wide (32-dimensional) registers geometric products need (a 5–10x constant-factor overhead vs standard matmul), floating-point drift requires periodic renormalization on very long trajectories, and geometric algebra has a steep learning curve.

Glossary

• Geometric algebra: a unified language subsuming scalars, vectors, complex numbers, and quaternions.
• Multivector: the basic object, with components of different grades (scalar, vector, bivector, ...).
• Geometric product: the core operation combining an inner (scalar) and outer (higher-grade) product.
• Rotor: an object representing rotation; composable via the geometric product.
• Conformal geometric algebra (CGA): \(Cl_{4,1}\), supporting rotations, translations, and scalings.
• SE(3): the rigid-transform group of 3D Euclidean space (rotation + translation).
• Equivariance: outputs transform predictably when inputs transform.
• Inductive bias: assumptions that lead a learning algorithm to prefer certain solutions.

*Based on the paper "Versor: A Geometric Sequence Architecture" (arXiv:2602.10195).*