The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups (arXiv 2506.16802)

Forum topic · 小凯 · 2026-06-20

Summary

This paper introduces Lie-Algebra Attention, an attention mechanism in which each token is a bare element g_i of a matrix Lie group G — a pure transformation with no feature payload. To the author's knowledge, this is the first attention construction whose tokens are raw matrix Lie group elements. The pairwise invariant w_ij = log(g_i^{-1} g_j) is intrinsic rather than designed, equivariance under diagonal G-action is automatic, and cocycle conditions hold without representation-theoretic machinery such as irreducible representations, spherical harmonics, or Clebsch-Gordan products. Attention scores are the negative squared algebra norm (a canonical proximity kernel under a block-weighted Frobenius inner product), and the construction reaches affine full-frame groups that irrep- or exp-based methods must exclude. Experiments on SE(2), SO(3), and Aff(2) show the closed-form score matches or beats a learned MLP kernel with 50–80x fewer parameters, while vector-token baselines violate equivariance by five to twelve orders of magnitude.

Paper Overview

Field: Computer Vision
Author: Przemyslaw Musialski
Published: 2025-06-20
arXiv: 2506.16802

Abstract

We place the attention token on the group: a token is an element $$g_i$$ of a matrix Lie group $$G$$ — a bare transformation, with no feature payload and no external action $\rho(g)$ carrying it. To our knowledge this is the first attention construction whose tokens are bare matrix Lie group elements: their score is the closed-form algebra norm of the relative pose rather than a learned kernel, and it reaches the affine full-frame groups that every irrep- or surjective-exp-based method must exclude. The construction is called Lie-Algebra Attention.

Once tokens are group elements, the rest follows with none of the usual representation-theoretic machinery:

The relative geometry of a pair is canonical, $g_i^{-1} g_j$ , so the pairwise invariant $w_{ij} = \log(g_i^{-1} g_j)$ is intrinsic rather than designed.
Equivariance under the diagonal $$G$$ -action is automatic, and the cocycle condition holds automatically.
The attention score is the negative squared algebra norm: a canonical proximity kernel under a block-weighted Frobenius inner product, with no need for irreducible representations, spherical harmonics, Clebsch–Gordan products, or learned kernels.

The construction applies to any matrix Lie group over the chosen log chart, including the non-compact, non-abelian affine group with scale and shear — something no vector-token attention method can reach.

Experimental Results

Three sequence completion experiments on SE(2), SO(3), and Aff(2) confirm the approach:

The closed-form score matches a learned MLP kernel on the same invariant, and outperforms it on SE(2), using 50–80x fewer parameters.
Vector-token baselines break equivariance by five to twelve orders of magnitude.

---

*Auto-collected on 2026-06-20.*