Physics-Informed Neural Networks (PINNs) are an appealing idea: approximate the solution of a PDE with a neural network, with a loss combining a data-fitting term and the residual of the governing equations. In practice, however, PINNs are notoriously hard to train. High-frequency and multi-scale solutions are almost never learned, and models often converge to trivial (e.g., zero) solutions. This is spectral bias: neural networks inherently prefer low-frequency functions, and learning of high-frequency components is essentially stalled.
Recently, it was observed that GAN-style adversarial training can significantly improve PINN training quality — the discriminator provides the generator (the PINN) with a more informative gradient signal. But nobody knew why.
Cao, So, Wang, and Wang now offer a theoretical explanation through the lens of the neural tangent kernel (NTK). The NTK describes the rate at which different frequency components of the network's output are learned during training. For PINNs, the NTK has a huge eigenvalue gap between low and high frequencies: large eigenvalues at low frequencies mean fast learning; near-zero eigenvalues at high frequencies mean no learning at all.
The effect of adversarial training is that the discriminator dynamically reweights the frequency components. When the PINN tends to fit only the low-frequency part, the discriminator generates a gradient signal that amplifies high-frequency errors — not by modifying the loss function, but by altering the eigenvalue distribution of the NTK in the training dynamics. This mechanism makes the PINN's NTK more balanced, giving high-frequency components a larger learning rate.
Based on this analysis, the authors propose a new, efficient training algorithm that does not require full GAN training — it only needs selective perturbation of the PINN's predictions to mimic the discriminator effect.
Open questions
- The NTK analysis is strictly valid in the infinite-width limit; for the small PINNs used in practice, the NTK approximation may deviate.
- Does the added stability from adversarial training hold uniformly across all PDE types?
- The paper claims accuracy improvements of "several orders of magnitude" — on which problems does this hold, and on which does it not?
References
1. Cao, Y., So, C. C., Wang, J., & Wang, H. (2026). *When and Why Adversarial Training Improves PINNs: A Neural Tangent Kernel Perspective*. arXiv:2605.15959 [cs.LG]. 2. Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). *Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations*. Journal of Computational Physics. 3. Jacot, A., Gabriel, F., & Hongler, C. (2018). *Neural Tangent Kernel: Convergence and Generalization in Neural Networks*. NeurIPS.