First-Order Stationarity of Reverse Diffusions: SDE-Based Sampling Guarantees for Diffusion Models

Forum topic · 小凯 · 2026-09-29

Summary

This arXiv paper (2609.31612) by Zhifeng Chen, Chenyang Jiang, and Yazhen Wang develops a first-order theory for diffusion models, paralleling the known connection between optimization and sampling. The authors prove that the SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates whenever the stationary potential of the forward process is strongly convex—a condition on the chosen noising process rather than on the data itself. This contraction property is unique to SDE-based reverse diffusion and does not hold for ODE-based reverse processes. The paper also incorporates discretization analysis and establishes averaged first-order stationarity bounds for samplers of both overdamped and underdamped diffusion models, providing the sampling analog of averaged gradient-norm guarantees in nonconvex optimization. These certificates are local and require no convexity assumptions: they guarantee consistency of the score rather than global mode weights. The work offers theoretical convergence guarantees relevant to the analysis and design of diffusion model samplers.

First-Order Stationarity of Reverse Diffusions

Research area: Machine Learning Authors: Zhifeng Chen, Chenyang Jiang, Yazhen Wang Published: 2026-09-25 arXiv: 2609.31612

Overview

Recent literature has revealed a deep connection between optimization and sampling. This paper develops the corresponding first-order theory for diffusion models, with two main contributions:

Key results

1. Exponential contraction for SDE-based reverse diffusion. The SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates, provided the stationary potential of the forward process is strongly convex. Importantly, this is a condition on the noising process one chooses, not on the data. The authors emphasize that this contraction is a unique advantage of SDE-based reverse diffusion and is absent in ODE-based reverse processes.

2. Discretization and stationarity bounds. Incorporating discretization analysis, the paper establishes averaged first-order stationarity bounds for samplers of both overdamped and underdamped diffusion models. These bounds are the sampling analog of averaged gradient-norm guarantees in nonconvex optimization.

Interpretation

As in nonconvex optimization, these convexity-free certificates are local in nature: they guarantee consistency of the score, rather than global mode weights.

Original abstract (excerpt)

> Recent literature has shown a strong connection between optimization and sampling. We develop the corresponding first-order theory for diffusion models. First, the SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates whenever the stationary potential of the forward process is strongly convex---a condition on the noising process one chooses, not on the data. This is a unique advantage of SDE-based reverse diffusion, absent in the reverse process based on ODEs. Second, we incorporate discretization and establish averaged first-order stationarity bounds---the sampling analog of averaged gradient-norm guarantees in nonconvex optimization---for samplers of both overdamped and underdamped diffusion models...

--- *Auto-collected on 2026-09-29.*