Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability of Reverse-Time Samplers

Forum topic · 小凯 · 2026-07-12

Summary

This paper (arXiv:2607.08757) shows that small score-matching error under the forward diffusion marginals does not guarantee numerical stability of discretized reverse-time samplers in diffusion models. The authors construct a single smooth score field with arbitrarily small forward-marginal L2 error whose learned reverse-time process is nonexplosive and arbitrarily close to the exact reverse-time process in path-space total variation, yet its Euler-Maruyama discretizations converge in probability while every positive moment diverges. Weak convergence can therefore hold even though every Wasserstein distance W_p (p >= 1) diverges. The same failure occurs within a single fixed finite neural architecture: a family of bounded, globally Lipschitz denoisers has vanishing forward-marginal error and path-space total variation distance while their Euler-Maruyama endpoints diverge in every W_p. As a positive result, projecting the learned denoiser onto a known bounded closed convex set containing the data support preserves pointwise accuracy, yields grid-consistent moment bounds, and gives Wasserstein convergence under mild local regularity. Experiments with small fixed DiT-style networks confirm these effects.

Paper Overview

Field: Machine Learning Author: Yiwei Zhou Published: 2026-07-09 arXiv: 2607.08757

Abstract

Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. We show that small forward-marginal error does not guarantee numerical stability. We construct a single smooth score field with arbitrarily small forward-marginal $$L^2$$ error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler--Maruyama discretizations converge in probability while every positive moment diverges. Thus weak convergence can hold even though every Wasserstein distance $$W_p$$ , $p\ge1$ , diverges. The same failure can occur within one fixed finite neural architecture. We construct a family of bounded, globally Lipschitz denoisers whose forward-marginal error and path-space total variation distance both tend to zero, while their Euler--Maruyama endpoints diverge in every $$W_p$$ . For compactly supported data, we also give a simple positive result. Projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, gives grid-consistent moment bounds, and yields Wasserstein convergence under mild local regularity. Experiments with small fixed DiT-style networks show large growth along rare numerical trajectories, and denoiser projection suppresses this growth while overall trajectory error remains small.

Key Contributions

Negative result: Arbitrarily small forward-marginal $$L^2$$ score error can coexist with Euler--Maruyama discretizations whose every positive moment diverges, even while the reverse-time process itself is nonexplosive and close to the exact one in path-space total variation.
Moment vs. weak convergence gap: Convergence in probability holds while every Wasserstein distance $$W_p$$ ( $p \ge 1$ ) diverges.
Architecture-level failure: The instability can be realized with a family of bounded, globally Lipschitz denoisers within one fixed finite neural architecture.
Positive result (projection fix): For compactly supported data, projecting the learned denoiser onto a bounded closed convex set containing the support preserves pointwise accuracy, gives grid-consistent moment bounds, and yields Wasserstein convergence under mild local regularity.
Experiments: Small fixed DiT-style networks exhibit score growth on rare numerical trajectories; projection suppresses this growth while overall trajectory error stays small.