Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention

Forum topic · 小凯 · 2026-08-21

Summary

A 2026 arXiv paper (2608.19171) by Sotirios P. Chatzis and Loukas Papadoulas introduces Lévy Attention, a cross-attention operator for deep models on irregularly-sampled time series that reports predictive uncertainty in a single forward pass at no extra cost. The operator's output is a stochastic integral against an inhomogeneous Poisson random measure: query-key compatibilities form an intensity over a continuous time-by-channel index space, and the measure scatters atoms whose interpolated values are averaged. In expectation it reduces to a mollified cosine-kernel attention, so it directly replaces softmax and trains with exact gradients. Unlike softmax, the Poisson construction preserves evidence (total compatibility mass) and disagreement (value-covariance trace) in closed form; an exact variance identity combines them into an RMS deviation estimate without any trained head. On t-PatchGNN, accuracy costs at most 5.6% versus matched controls and is free on the sparsest datasets. The free disagreement signal beats 20-round MC dropout across five matched suites, its calibrated Gaussian achieves zero-shot CRPS better than fifty-sample baselines, split-conformal wrappers hit nominal coverage at every horizon, and one 1.4-second pass ranks 3,383 unseen patients by trustworthiness.

Overview

Field: Machine Learning
Authors: Sotirios P. Chatzis, Loukas Papadoulas
Published: 2026-08-19
arXiv: 2608.19171

Key Points

Deep models for irregularly-sampled time series answer queries at arbitrary continuous timestamps but report nothing about how trustworthy each answer is. The paper shows the attention layer itself can close this gap: with the right stochastic formulation, the forward pass that makes each prediction also reports, in closed form and at no extra cost, how far it should be trusted.
Lévy Attention is a cross-attention operator whose output is a stochastic integral against an inhomogeneous Poisson random measure: query-key compatibilities assemble an intensity over a continuous (time × channel) index space, the measure scatters atoms under it, and the output averages an interpolated value field at those atoms.
In expectation it reduces to a mollified cosine-kernel attention, so it replaces the softmax layer and trains with exact gradients.
What softmax discards, the Poisson construction preserves in closed form: the evidence Λ_q (total compatibility mass) and the disagreement trΣ_V(q) (value covariance trace). An exact variance identity combines them into σ̂(q) = √(trΣ_V(q) φ(Λ_q)), the RMS deviation of the sampled operator, emitted by the deterministic forward pass with no trained head.
Empirically, disagreement carries signal, while the evidence factor ranges from uninformative on dense data to strongly informative on sparse data.
On t-PatchGNN, the operator swap costs at most 5.6% accuracy against matched controls, and is free on the sparsest datasets.
The free disagreement signal outperforms 20 rounds of MC dropout across five matched suites; the σ̂-scaled calibrated Gaussian achieves zero-shot CRPS better than fifty-sample baselines; split-conformal wrappers reach nominal coverage at every horizon; and a single forward pass ranks 3,383 unseen patients by trustworthiness in 1.4 seconds.

Notes

Automatically collected on 2026-08-21.