The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups

Forum topic · 小凯 · 2026-06-20

Summary

A new arXiv paper (2506.16802) by Przemyslaw Musialski introduces Lie-Algebra Attention, an attention mechanism in which each token is a bare element of a matrix Lie group—a pure transformation with no feature payload or external representation acting on it. Because tokens are group elements, pairwise geometry is canonical: the relative pose g_i^{-1} g_j yields an intrinsic invariant w_ij = log(g_i^{-1} g_j), and equivariance under diagonal group action holds automatically. Attention scores are closed-form negative squared algebra norms (a canonical proximity kernel under a block-weighted Frobenius inner product), requiring no irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernels. The construction applies to any matrix Lie group, including the non-compact, non-Abelian affine group with scale and shear, which irrep- or exp-based methods must exclude. Experiments on SE(2), SO(3), and Aff(2) show the closed-form scores match or beat learned MLP kernels with 50–80x fewer parameters, while vector-token baselines break equivariance by five to twelve orders of magnitude.

Paper Overview

Field: Computer Vision
Author: Przemyslaw Musialski
Published: 2025-06-20
arXiv: 2506.16802

Abstract

We place the attention token on the group: a token is an element $$g_i$$ of a matrix Lie group $$G$$ — a bare transformation, with no feature payload and no external action $\rho(g)$ carrying it. To our knowledge this is the first attention construction whose tokens are bare matrix Lie group elements: their score is the closed-form algebra norm of the relative pose rather than a learned kernel, and it reaches the affine full-frame groups that every irrep- or surjective-exp-based method must exclude. The authors call this construction Lie-Algebra Attention.

Once tokens are group elements, the rest follows without the usual representation-theoretic machinery:

The relative geometry of a pair is canonical, $g_i^{-1} g_j$ , so the pairwise invariant $w_{ij} = \log(g_i^{-1} g_j)$ is intrinsic rather than designed.
Equivariance under the diagonal $$G$$ -action holds automatically, and the cocycle condition is satisfied by construction.
The attention score is a negative squared algebra norm: the canonical proximity kernel under a block-weighted Frobenius inner product, with no irreducible representations, spherical harmonics, Clebsch–Gordan products, or learned kernels.

The construction applies to any matrix Lie group whose chosen logarithm contains the relative pose, including the non-compact, non-Abelian affine group with scale and shear — something no vector-token attention method can reach.

Experiments

Three sequence completion experiments on $$SE(2)$$ , $$SO(3)$$ , and $$Aff(2)$$ confirm the claims:

The closed-form scores match learned MLP kernels on the same invariants, and outperform them on $$SE(2)$$ , using 50–80× fewer parameters.
Vector-token baselines break equivariance by five to twelve orders of magnitude.

---

*Source: arXiv:2506.16802*