Learning Physics from an Imperfect Ancestor: Distilling Neural Operator Priors into PINNs

Forum topic · 小凯 · 2026-09-23

Summary

This arXiv paper (2609.24947) by Mousavi, Kadeethum, Bouklas, and Goswami addresses two common failure modes in neural PDE solvers: neural operators (NOs) degrade sharply outside their training distribution, while physics-informed neural networks (PINNs) can converge to physically incorrect solutions when the governing residual admits multiple states. The authors show these failures can be jointly resolved: an imperfect NO supplies a structural prior that places a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. Their three-stage framework freezes a physics-informed NO's spatial basis, extrapolates its solution branch to out-of-distribution parameters via a polynomial continuation prior, then distills the resulting field into a fresh PINN. On three nonlinear PDEs—1D viscous Burgers, 2D steady Allen-Cahn near a pitchfork bifurcation, and 2D lid-driven cavity flow—standard PINNs collapse to trivial branches where distillation from a coarse extrapolated operator recovers the correct nontrivial solution. For cavity flow at Re=3200, extrapolation accelerated convergence with competitive accuracy using fewer parameters and optimization steps than recent baselines.

Paper Overview

Field: Machine Learning Authors: S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas, Somdatta Goswami Published: 2026-09-21 arXiv: 2609.24947

Abstract

Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed neural networks avoid dependence on labeled data, yet their optimization can be basin-fragile: when the governing residual admits multiple solutions, a PINN trained from scratch may converge to a physically incorrect state despite achieving a small residual. We show that these failure modes can be addressed jointly: an imperfect NO provides the structural prior needed to place a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. We introduce a three-stage framework that freezes the spatial basis of a physics-informed NO, extrapolates its solution branch to an out-of-distribution parameter via a polynomial continuation prior, and then distills the resulting field into a fresh PINN. The NO does not need to be accurate at the target; it transmits branch information, and PDE residual minimization within the PINN governs convergence.

Evaluation

The framework is evaluated on three nonlinear PDEs:

1D viscous Burgers equation
2D steady Allen-Cahn equation near a pitchfork bifurcation: the trivial solution exactly satisfies the PDE residual, so a standard PINN collapses to the trivial zero branch, while distillation from a coarse extrapolated operator recovers the nontrivial branch consistent with a finite-difference reference
2D steady lid-driven cavity flow: extrapolation to Reynolds number Re=3200 accelerates convergence to the correct physical state, achieving competitive accuracy with fewer parameters and optimization steps than recent literature baselines

Key Takeaway

A simple principle: a neural operator does not need to predict the solution accurately to be useful—it only needs to identify the correct basin, from which PINN optimization can recover the solution.

---

*Auto-collected on 2026-09-23*