Paper Overview
Field: Machine Learning Authors: Zhifeng Chen, Chenyang Jiang, Yazhen Wang Published: 2026-09-25 arXiv: 2609.31612
Summary
Recent literature has revealed a deep connection between optimization and sampling. This paper develops the corresponding first-order theory for diffusion models, with two main contributions:
1. Exponential Contraction for SDE-Based Reverse Flows
The SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates, provided the stationary potential of the forward process is strongly convex. Key aspects of this result:
- The strong convexity condition applies to the noising process one chooses, not to the data itself.
- This is a unique advantage of SDE-based reverse diffusion and is absent in ODE-based reverse processes.
2. Averaged First-Order Stationarity Bounds with Discretization
The authors incorporate discretization analysis and establish averaged first-order stationarity bounds for samplers of both overdamped and underdamped diffusion models. These bounds are the sampling analog of averaged gradient-norm guarantees in nonconvex optimization.
As in nonconvex optimization, this certificate requires no convexity assumptions and is inherently local: it guarantees the consistency of the score, rather than global mode weights.
Original Abstract (excerpt)
> Recent literature has shown a strong connection between optimization and sampling. We develop the corresponding first-order theory for diffusion models. First, the SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates whenever the stationary potential of the forward process is strongly convex---a condition on the noising process one chooses, not on the data. This is a unique advantage of SDE-based reverse diffusion, absent in the reverse process based on ODEs. Second, we incorporate discretization and establish averaged first-order stationarity bounds---the sampling analog of averaged gradient-norm guarantees in nonconvex optimization---for samplers of both overdamped and underdamped diffusion models...
--- *Auto-collected on 2026-09-29*