Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability of Reverse-Time Samplers

Forum topic · 小凯 · 2026-07-11

Summary

A paper by Yiwei Zhou (arXiv:2507.08175) shows that small score-matching error under forward diffusion marginals does not guarantee numerical stability of discretized reverse-time samplers. The authors construct a single smooth score field with arbitrarily small forward-marginal L2 error whose learned reverse-time process is nonexplosive, has finite moments of every order, and approximates the exact reverse-time process in path-space total variation. Yet its Euler-Maruyama discretizations converge only in probability while every positive moment diverges, so weak convergence holds even as every Wasserstein distance W_p (p >= 1) diverges. The failure persists within a single fixed finite neural architecture via a family of bounded, globally Lipschitz denoisers. A positive result is also given: projecting the learned denoiser onto a known bounded closed convex set containing compactly supported data preserves pointwise accuracy and yields uniform moment bounds and Wasserstein convergence under mild local regularity.

论文概要

研究领域: Machine Learning (ML) 作者: Yiwei Zhou 发布时间: 2026-07-10 arXiv: 2507.08175

Abstract

Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. This paper shows that small forward-marginal error does not guarantee numerical stability.

Key points

The authors construct a single smooth score field with arbitrarily small forward-marginal L² error.
The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation.
However, its Euler–Maruyama discretizations converge in probability while every positive moment diverges.
Consequently, weak convergence can hold even though every Wasserstein distance W_p (p ≥ 1) diverges.
The same failure can occur within one fixed finite neural architecture: a family of bounded, globally Lipschitz denoisers is constructed whose forward-marginal error and path-space total variation distance both tend to zero, while their Euler–Maruyama endpoints diverge in every W_p.
Positive result: for compactly supported data, projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, gives grid-uniform moment bounds, and yields Wasserstein convergence under mild local regularity.