Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Irregularly-Sampled Time Series

Forum topic · 小凯 · 2026-08-21

Summary

Lévy Attention (arXiv:2608.19171) by Sotirios P. Chatzis and Loukas Papadoulas introduces a cross-attention operator for deep models on irregularly sampled time series that reports predictive uncertainty in the same forward pass that makes each prediction, at no extra cost. The output is a stochastic integral against an inhomogeneous Poisson random measure: query-key compatibilities form an intensity over a continuous (time × channel) index space, the measure scatters atoms, and the output averages an interpolated value field at those atoms. In expectation it reduces to mollified cosine-kernel attention, replacing the softmax layer and training with exact gradients. The Poisson construction retains in closed form what softmax discards: evidence Λ_q (total compatibility mass) and divergence trΣ_V(q) (value spread). An exact variance identity combines them into σ̂(q), the root-mean-square deviation of the sampled operator, emitted deterministically without a trained head. Experiments on t-PatchGNN show at most 5.6% accuracy cost versus matched controls, zero cost on the sparsest datasets; the free divergence signal outperforms 20× MC dropout, σ̂-scaled Gaussians beat 50-sample CRPS zero-shot, and split-conformal wrapping attains nominal coverage at every level, ranking 3,383 unseen patients by trustworthiness in one 1.4-second pass.

Paper Overview

Field: Machine Learning Authors: Sotirios P. Chatzis, Loukas Papadoulas Published: 2026-08-19 arXiv: 2608.19171

Abstract (English)

Deep models for irregularly-sampled time series answer queries at arbitrary continuous timestamps, yet report nothing about how far each answer should be trusted. The authors show the attention layer itself can close that gap: with the right stochastic formulation, the pass that makes each prediction also reports, in closed form and at no extra cost, how far it should be trusted.

Key Idea: Lévy Attention

Lévy Attention is a cross-attention operator whose output is a stochastic integral against an inhomogeneous Poisson random measure:

Query-key compatibilities assemble an intensity over a continuous (time × channel) index space.
The measure scatters atoms under this intensity.
The output averages an interpolated value field at those atoms.

In expectation it reduces to a mollified cosine-kernel attention, so it replaces the softmax layer and trains with exact gradients.

Closed-Form Uncertainty

What softmax discards, the Poisson construction retains in closed form:

Evidence Λ_q — total compatibility mass
Divergence trΣ_V(q) — spread of the value distribution

An exact variance identity combines them into

σ̂(q) = √(trΣ_V(q) · φ(Λ_q))

which is the root-mean-square deviation of the sampled operator, emitted by the deterministic forward pass — no trained head required.

Empirical Findings

Divergence carries signal; the evidence factor ranges from uninformative on dense data to strongly informative on sparse data.
On t-PatchGNN, replacing the attention operator costs at most 5.6% accuracy versus matched controls, and zero cost on the sparsest datasets.
The free divergence signal outperforms 20× MC dropout on matched five-suite benchmarks.
σ̂-scaled calibrated Gaussians achieve zero-shot CRPS beating fifty-sample baselines.
Split-conformal wrapping reaches nominal coverage at every level.
A single forward pass ranks 3,383 unseen patients by trustworthiness in 1.4 seconds.