Summary
This arXiv paper (2608.11173) by Eric A. F. Reinhardt and Adam J. Hauser presents a quantum computing roadmap for realizing softmax attention, the core mechanism of Transformers, exactly on quantum hardware. For problems where inputs and outputs lie on the probability simplex, each component of softmax attention maps to a precise quantum analog: attention scores become Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs; the exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary yields sparse attention with exact zeros at finite parameter values. Softmax temperature corresponds to a repetition count via post-selected measurement rounds, value aggregation to a deterministic column-stochastic channel, gated residuals to a single auxiliary qubit preparation angle, and each learnable parameter to a rotation gate angle. Composite layers are exact in the infinite-sampling limit, and a fully coherent variant is epsilon-approximate in the infinite-depth limit via quantum singular value transformation. The algebraic core is machine-verified in Lean 4.
Paper Overview
- Field: Machine Learning
- Authors: Eric A. F. Reinhardt, Adam J. Hauser
- Published: 2026-08-11
- arXiv: 2608.11173
Abstract
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization.
Key correspondences established in the paper:
- Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs.
- The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection; its boundary expresses sparse attention with exact zeros at finite parameter values.
- Softmax temperature is a repetition count: post-selected measurement rounds realize the discretized inverse temperature exactly.
- Value aggregation is a deterministic column-loading channel extending column-stochastic value matrices.
- Gated residual is a single auxiliary-qubit preparation angle, with an additive identity at mixing angle π/2.
- Every learnable parameter is a rotation gate angle.
Composite layers are exact in the infinite-sampling limit, with each attention score corresponding to one measure-and-reload step; a fully coherent variant is ε-approximate in the infinite-depth limit via quantum singular value transformation. The algebraic core is machine-verified in Lean 4.
---
*Auto-collected on 2026-08-13.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633407