The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups

Forum topic · 小凯 · 2026-06-23

Summary

This arXiv paper (2506.18493) by Przemyslaw Musialski proposes Lie-Algebra Attention, an attention mechanism where each token is a bare element of a matrix Lie group — a raw transformation with no feature payload or external group action. The attention score is a closed-form algebra norm of relative pose, s_ij = -||log(g_i^{-1} g_j)||^2 / tau, replacing learned kernels and avoiding irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernel functions. Pairwise invariants log(g_i^{-1} g_j) are intrinsic rather than designed, equivariance under diagonal group action becomes tautological, and the cocycle condition holds automatically. The construction applies to any matrix Lie group with a log map, including non-compact non-Abelian affine groups (Aff) containing scale and shear, which irrep- or surjective-exp-based methods must exclude. Experiments on sequence completion tasks in SE(2), SO(3), and Aff(2) show the closed-form scores match learned MLP kernels on the same invariants and outperform them on SE(2) with 50-80x fewer score parameters, while vector-token baselines break invariance by five to twelve orders of magnitude.

Overview

Field: Computer Vision Author: Przemyslaw Musialski Published: 2025-06-23 arXiv: 2506.18493

Key Idea

The paper places the attention token on the group itself: a token is an element g_i of a matrix Lie group G — a bare transformation, with no feature payload and no external action rho(g) carrying it. To the author's knowledge, this is the first attention construction whose tokens are bare matrix Lie group elements.

Method: Lie-Algebra Attention

Once tokens are group elements, the rest follows without the usual representation-theoretic machinery:

The relative geometry of a pair is canonical, g_i^{-1} g_j, so the pairwise invariant w_{ij} = log(g_i^{-1} g_j) is intrinsic rather than designed.
Equivariance under the diagonal G-action is tautological; the cocycle condition holds automatically.
The attention score is a negative squared algebra norm:

s_{ij} = -||log(g_i^{-1} g_j)||_λ^2 / τ

a canonical proximity kernel under a block-weighted Frobenius inner product — no irreducible representations, spherical harmonics, Clebsch-Gordan products, or learned kernels.

Generality

The construction applies to any matrix Lie group with a log map, including non-compact non-Abelian affine groups (containing scale and shear) — the full-frame groups that every irrep- or surjective-exp-based method must exclude.

Experiments

Three sequence-completion experiments on SE(2), SO(3), and Aff(2) confirm:

Closed-form scores match learned MLP kernels on the same invariants, and outperform them on SE(2).
The approach uses 50–80× fewer score parameters.
Vector-token baselines break invariance by five to twelve orders of magnitude.

---

*Auto-collected on 2026-06-23*