The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups

Forum topic · 小凯 · 2026-06-23

Summary

This arXiv paper (2506.18493) by Przemyslaw Musialski introduces Lie-Algebra Attention, an attention mechanism whose tokens are bare elements g_i of a matrix Lie group G, with no feature payload or external representation action. To the author's knowledge, it is the first attention construction using bare matrix Lie group elements as tokens: attention scores are the closed-form algebra norm of relative pose rather than a learned kernel, s_ij = -||log(g_i^{-1} g_j)||^2 / tau, a canonical proximity kernel under a block-weighted Frobenius inner product. The pairwise invariant w_ij = log(g_i^{-1} g_j) is intrinsic rather than designed, and equivariance under diagonal G-actions is tautological, with the cocycle condition holding automatically. The method requires no irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernels, and uniquely covers non-compact non-abelian affine groups (with scale and shear) that irrep- or exp-based methods must exclude. Experiments on sequence completion in SE(2), SO(3), and Aff(2) show the closed-form scores match learned MLP kernels on the same invariants and outperform them on SE(2) with 50-80x fewer score parameters, while vector-token baselines violate invariance by five to twelve orders of magnitude.

Paper Overview

Field: Computer Vision Author: Przemyslaw Musialski Published: 2025-06-23 arXiv: 2506.18493

Summary

This paper puts the attention token on the group: a token is an element g_i of a matrix Lie group G — a bare transformation, with no feature payload and no external action rho(g) carrying it. According to the author, this is the first attention construction whose tokens are bare matrix Lie group elements: their score is the closed-form algebra norm of the relative pose rather than a learned kernel, and it reaches the affine full-frame groups that every irrep- or surjective-exp-based method must exclude. The construction is called Lie-Algebra Attention.

Once tokens are group elements, the rest follows without any of the usual representation-theoretic machinery:

The relative geometry of a pair is canonical, g_i^{-1} g_j, so the pairwise invariant w_{ij} = log(g_i^{-1} g_j) is intrinsic rather than designed.
Equivariance under the diagonal G-action is tautological, and the cocycle condition holds automatically.
The attention score is the negative squared algebra norm: s_{ij} = -||log(g_i^{-1} g_j)||_λ^2 / τ — a canonical proximity kernel under a block-weighted Frobenius inner product, with no irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernels.

The construction applies to any matrix Lie group whose logarithm contains the relative pose, including non-compact non-abelian affine groups (with scale and shear), which no vector-token attention method can reach.

Experimental Results

Three sequence completion experiments on SE(2), SO(3), and Aff(2) confirm the claims:

The closed-form scores match learned MLP kernels on the same invariants, and outperform them on SE(2), using 50 to 80 times fewer score parameters.
Vector-token baselines break invariance by five to twelve orders of magnitude.

Original Abstract (excerpt)

> We place the attention token on the group: a token is an element g_i of a matrix Lie group G — a bare transformation, with no feature payload and no external action rho(g) carrying it. To our knowledge this is the first attention construction whose tokens are bare matrix Lie group elements: their score is the closed-form algebra norm of the relative pose rather than a learned kernel, and it reaches the affine full-frame groups that every irrep- or surjective-exp-based method must exclude. We call it Lie-Algebra Attention. Once tokens are group elements, the rest follows with none of the usual representation-theoretic machinery. The relative geometry of a pair is canonical, g_i^{-1} g_j, so the pairwise invariant w_{ij} = log(g_i^{-1} g_j) is intrinsic rather than designed; equivariance un...

--- *Auto-collected on 2026-06-23*

The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups

Summary

Paper Overview

Summary

Experimental Results

Original Abstract (excerpt)

Tags