Neural Networks Finally Learn to Converge: A Decade-Awaited Mathematical Proof for DGM and PINN Global Convergence

Forum topic · 小凯 · 2026-07-28

Summary

A new paper by Justin Sirignano, Konstantinos Spiliopoulos, and Samuel Cohen (arXiv:2607.24726) proves global convergence of the Deep Galerkin Method (DGM) and Physics-Informed Neural Networks (PINNs) for solving a class of semilinear partial differential equations (PDEs). For nearly a decade, PINNs and DGM have been widely used to approximate PDE solutions—replacing costly grid-based methods like finite elements—but practitioners routinely observed training that stalled or oscillated, since neural network loss landscapes are non-convex and gradient descent can land in spurious local minima that do not satisfy the PDE. The new proof shows that, for semilinear PDEs, gradient descent on the PDE residual loss converges to the true solution given sufficiently over-parameterized (wide) networks and sufficient training. The argument combines the universal approximation theorem, analysis of the loss landscape showing gradient descent avoids bad local minima, over-parameterization theory, Neural Tangent Kernel (NTK) dynamics, and dynamical systems stability theory. The result provides the first rigorous convergence guarantee for a major class of nonlinear PDEs, strengthening the theoretical foundations of scientific machine learning, though quasilinear PDEs such as Navier-Stokes remain open challenges.

Overview

A long-awaited theoretical result has arrived: global convergence of DGM and PINN algorithms for solving nonlinear PDEs, proving that neural network solvers converge to the true PDE solution rather than getting stuck in spurious local minima.

Paper info:

Title: Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs
Authors: Justin Sirignano, Konstantinos Spiliopoulos, Samuel Cohen
arXiv: 2607.24726
Categories: cs.LG, math.NA
Posted: 2026-07-27

Key points

The problem: For nearly ten years, Physics-Informed Neural Networks (PINNs) and the Deep Galerkin Method (DGM) have been used to solve PDEs without grids, but neural network loss functions are non-convex. Gradient descent can converge to local minima where the loss is nonzero—and such minima do not correspond to solutions of the PDE. Practitioners frequently saw training stall or oscillate.
The result: The authors prove a rigorous theorem: for a class of semilinear PDEs (linear in the highest-order derivatives, possibly nonlinear in lower-order terms, e.g. reaction–diffusion equations u_t = Δu + f(u)), gradient descent training of a neural network on the PDE residual loss converges to the true PDE solution, provided the network is sufficiently wide and training is sufficiently long.
Why it matters: This is the first rigorous guarantee that PINN/DGM training actually works for a meaningful nonlinear PDE class, closing a major theoretical gap in scientific machine learning (SciML).

Main ideas behind the proof

1. Universal approximation — a network exists whose output is arbitrarily close to the true solution u*. 2. Loss landscape structure — for semilinear PDEs, the PDE's own structure (uniqueness, regularity of solutions) is encoded in the loss, so gradient descent does not get trapped in "bad" local minima. 3. Over-parameterization — very wide networks smooth the loss landscape, reducing bad minima; the authors prove gradient descent converges to global optimality with high probability in this regime. 4. Neural Tangent Kernel (NTK) view — when the network is very wide, training dynamics become highly smooth, so gradient descent reliably reaches the minimum, which corresponds to the true PDE solution. 5. Dynamical systems stability — for certain PDEs, the solution space has an attractor structure that the network's training dynamics are "compatible" with, pulling training toward the correct solution.

Scope and limitations

The proof covers semilinear PDEs only. Quasilinear and fully nonlinear PDEs—e.g. the Navier–Stokes equations with their nonlinear convection term u · ∇u—remain open challenges.
Practical issues (training efficiency, architecture design, hyperparameter tuning) are unaffected, but the core fear—"I may just be wasting time in a local minimum"—can now be set aside for this PDE class.

Significance

PINNs/DGM can now be used for semilinear PDEs with confidence: with a wide enough network and enough training, convergence to the correct solution is guaranteed.
The proof bridges several traditionally separate fields: PDE regularity theory, neural network approximation theory, gradient descent convergence, and dynamical systems stability.
It provides a strong theoretical signal that "black-box + physics constraints" approaches in SciML are indeed reliable under identifiable conditions.

Reference

Sirignano, J., Spiliopoulos, K., & Cohen, S. (2026). *Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs*. arXiv:2607.24726