Overview
Field: Machine Learning Authors: Eric A. F. F. Reinhardt, Adam J. Hauser Published: 2026-08-11 arXiv: 2608.11173
Abstract
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization.
Key Mechanisms
- Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs.
- Exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection. Its boundary expresses sparse attention with exact zeros at finite parameter values.
- Softmax temperature is a repetition count: post-selected measurement rounds realize discretized inverse temperature exactly.
- Value aggregation is a deterministic column-loading channel extending column-stochastic value matrices.
- Gated residual is a single auxiliary qubit's preparation angle, with an additive identity at mixing angle π/2.
- Each learnable parameter is a rotation gate angle.
- Composite layers are exact in the infinite-sampling limit, with one measure-and-reload step per attention score.
- Fully coherent variants are ε-approximate in the infinite-depth limit via quantum singular value transformation.
- The algebraic core is machine-verified in Lean 4.