A Unified Compressed Sensing View of Fourier Neural Operators and 3D Gaussian Splatting

Forum topic · 小凯 · 2026-08-26

Summary

This post argues that compressed sensing (Candès, Romberg & Tao, 2006; Donoho, 2006) provides a unified mathematical framework for two seemingly unrelated techniques: the Fourier Neural Operator (FNO) for PDE solving and 3D Gaussian Splatting (3DGS) for real-time radiance field rendering. Both methods, the author contends, find a basis in which the physical world is extremely sparse and reconstruct continuous infinite-dimensional fields from few incoherent measurements. FNO uses a global frequency-domain Fourier basis, truncating to a small set of low-frequency modes so that a coarse spatial grid suffices for zero-shot super-resolution prediction. 3DGS uses local anisotropic Gaussian primitives concentrated on object surfaces, with multi-view 2D projections acting as Fourier-slice incoherent measurements. Their reconstruction stages also correspond: FNO operator learning distorts iterative L1-style optimization into a single forward pass, while 3DGS adaptive density control (clone/split/prune) mirrors support-set recovery in matching pursuit. The post proposes future hybrids such as Gaussian Neural Operators to remove Gibbs oscillations near shocks, and FNO-driven 4D Gaussian dynamics for physically consistent 100+ FPS interactive simulation. References include Li et al. (ICLR 2021) and Kerbl et al. (SIGGRAPH 2023).

This post from zhichai.net presents a theoretical unification of compressed sensing (CS), the Fourier Neural Operator (FNO), and 3D Gaussian Splatting (3DGS). Its central claim: both FNO and 3DGS instantiate the same CS structure — find a basis in which a complex physical field is extremely sparse, then reconstruct the continuous infinite-dimensional field from very few incoherent observations at low computational cost.

Key points

1. The CS triangle: sparsity, incoherent measurement, nonlinear recovery

Classical Nyquist sampling demands grid/voxel counts scaling with the highest signal frequency.
CS theory (Candès, Romberg & Tao; Donoho, 2006) shows that if a signal is sparse in some basis Ψ (K ≪ N nonzero coefficients), it can be recovered from far fewer incoherent measurements Φ (satisfying RIP) via convex/learned optimization:

\min \|\theta\|_1 \quad \text{s.t.} \quad y = \Phi \Psi \theta

Incoherence between Φ and Ψ determines how few measurements are needed.

2. Choice of sparse basis: global frequency vs. local space

| CS dimension | FNO | 3DGS | | :--- | :--- | :--- | | Target field | PDE solution u(x) (velocity, pressure) | 3D radiance field f(x) (geometry + color) | | Sparse basis Ψ | Global complex Fourier basis {e^{ik·x}} | Local anisotropic Gaussians G(x; μᵢ, Σᵢ) | | Sparsity structure | Energy concentrated in low frequencies (k ≤ k_max) | Mass concentrated on thin surface layers | | Effect | Tens of Fourier modes summarize global vortex evolution | Millions of Gaussian primitives replace trillions of empty voxels |

FNO achieves frequency-domain global sparsity; 3DGS achieves spatial-domain local sparsity — two opposite routes to the same compression goal.

3. Measurement matrices as incoherent projections

FNO: sampling the solution on a coarse 64×64 spatial grid is a spatial impulse measurement, maximally incoherent with the frequency basis — enabling zero-shot generalization to 1024×1024.
3DGS: multi-view 2D photographs are line-integral (Fourier-slice) measurements of the 3D spectrum; a dozen views suffice to recover millions of Gaussian means μᵢ and covariances Σᵢ.

4. Reconstruction algorithms

FNO: operator learning condenses the iterative CS optimization into a single forward pass with complex convolutions, solving PDEs in milliseconds.
3DGS: adaptive density control (clone / split / prune by gradient statistics) mathematically parallels support-set recovery in orthogonal matching pursuit.

5. Proposed future fusion

Gaussian Neural Operator (GNO): use spatially local Gaussian bases to eliminate Gibbs oscillations that FNO exhibits near irregular surfaces and shocks.
Physics-constrained 4D Gaussians: use FNO-computed Navier-Stokes low-frequency dynamics as a prior driving 3D Gaussian space-time evolution, for physically faithful 100+ FPS interactive simulation.

References cited

Candès, E. J., Romberg, J., & Tao, T. (2006). *Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information*. IEEE Transactions on Information Theory, 52(2), 489-509.
Donoho, D. L. (2006). *Compressed sensing*. IEEE Transactions on Information Theory, 52(4), 1289-1306.
Li, Z., et al. (2021). *Fourier Neural Operator for Parametric Partial Differential Equations*. ICLR 2021 (Oral). arXiv:2010.08895.
Kerbl, B., et al. (2023). *3D Gaussian Splatting for Real-Time Radiance Field Rendering*. ACM Transactions on Graphics (SIGGRAPH 2023), 42(4), 139.