Paper: Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability of Reverse-Time Samplers

Forum topic · 小凯 · 2026-07-11

Summary

A new arXiv paper (2507.08175) by Yiwei Zhou shows that small forward-marginal score-matching error does not guarantee numerical stability of discretized reverse-time diffusion samplers. The author constructs a single smooth score field whose forward-marginal L² error is arbitrarily small; the resulting learned reverse-time process is nonexplosive, has finite moments of every order, and is arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler–Maruyama discretizations converge in probability while every positive moment diverges—so weak convergence can hold even as every Wasserstein distance W_p (p≥1) diverges. The same failure can occur within a single fixed finite neural architecture, demonstrated via a family of bounded, globally Lipschitz denoisers whose forward-marginal and path-space total variation errors vanish while their Euler–Maruyama endpoints diverge in every W_p. For compactly supported data, a positive result shows that projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, yields grid-uniform moment bounds, and gives Wasserstein convergence under mild local regularity.

Paper Overview

Field: Machine Learning Author: Yiwei Zhou Published: 2026-07-10 arXiv: 2507.08175

Abstract (Original)

Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. We show that small forward-marginal error does not guarantee numerical stability. We construct a single smooth score field with arbitrarily small forward-marginal L² error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler–Maruyama discretizations converge in probability while every positive moment diverges. Thus weak convergence can hold even though every Wasserstein distance W_p, p≥1, diverges. The same failure can occur within one fixed finite neural architecture. We construct a family of bounded, globally Lipschitz denoisers whose forward-marginal error and path-space total variation distance both tend to zero, while their Euler–Maruyama endpoints diverge under every W_p. For compactly supported data, we also give a simple positive result: projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, yields grid-uniform moment bounds, and produces Wasserstein convergence under mild local regularity.

Key Takeaways

Small forward-marginal score error does not certify numerical stability of reverse-time samplers.
There exists a single smooth score field with arbitrarily small forward-marginal L² error whose Euler–Maruyama discretizations converge in probability while every positive moment diverges.
Weak convergence can coexist with divergence of every Wasserstein distance W_p (p ≥ 1).
The failure mode can occur even within a fixed finite neural architecture (bounded, globally Lipschitz denoisers).
Positive result: for compactly supported data, projecting the learned denoiser onto a bounded closed convex set containing the support preserves pointwise accuracy and yields Wasserstein convergence under mild local regularity.