Compressed Sensing as a Unifying Lens: Fourier Neural Operator and 3D Gaussian Splatting

Forum topic · 小凯 · 2026-08-26

Summary

This article argues that Compressed Sensing (CS) theory provides a unified mathematical framework for two seemingly unrelated techniques: the Fourier Neural Operator (FNO) for solving partial differential equations and 3D Gaussian Splatting (3DGS) for radiance field rendering. It reviews CS's three pillars—sparsity priors, incoherent measurement, and nonlinear sparse recovery—and maps them onto both systems. FNO exploits a global Fourier basis where PDE solution energy concentrates in low-frequency modes, while 3DGS uses local anisotropic Gaussians concentrated on object surfaces, both achieving sparse representations of infinite-dimensional continuous fields. Measurement processes are analyzed as duals: FNO samples solutions on coarse spatial grids (incoherent with frequency bases, enabling zero-shot super-resolution), while 3DGS captures multi-view 2D projections (Fourier slice theorem). Recovery methods are compared: FNO amortizes iterative optimization into a single forward network pass, while 3DGS's adaptive density control (clone/split/prune) parallels orthogonal matching pursuit for support-set recovery. The article proposes future hybrids: Gaussian neural operators using spatially local bases to avoid Gibbs oscillations, and physics-constrained 4D Gaussian evolution driven by FNO-solved Navier-Stokes dynamics for real-time physically faithful simulation.

Compressed Sensing as a Unifying Lens: Fourier Neural Operator (FNO) and 3D Gaussian Splatting (3DGS)

This forum post draws an analogy: imagine a 100-million-pixel painting. Photographing every pixel is infeasible, yet since only a few thousand strokes carry the actual information, a few random glances suffice to reconstruct the whole. This is Compressed Sensing (CS)—and the post argues that FNO and 3DGS share one mathematical structure under it: both seek a basis that makes the physical world extremely sparse, then reconstruct infinite-dimensional continuous fields from very few incoherent observations at low computational cost.

Key points

1. The core triangle of compressed sensing

Classical Nyquist sampling requires sampling above twice the highest frequency, forcing massive grids/voxels. CS (Candès, Romberg, Tao; Donoho) shows that if a signal is sparse in some basis $\mathbf{\Psi}$ , it can be recovered from far fewer incoherent measurements $\mathbf{\Phi}$ :

Sparsity prior: the signal has only K nonzero coefficients in basis Ψ (K ≪ N)
Incoherence: measurement matrix Φ and sparsity basis Ψ satisfy the Restricted Isometry Property (RIP)
Nonlinear recovery: solving an underdetermined inverse problem via convex optimization or neural networks: min ‖θ‖₁ s.t. y = Φ Ψ θ

2. Sparse basis selection: frequency-domain vs. spatial-domain

Both systems decompose $\mathbf{x} = \mathbf{\Psi}\boldsymbol{\theta}$ with $\|\boldsymbol{\theta}\|_0 = K \ll N$ , but choose opposite bases:

| CS dimension | FNO | 3DGS | | :--- | :--- | :--- | | Target | Continuous PDE solution field $$u(x)$$ (velocity, pressure) | Continuous radiance field $f(\mathbf{x})$ (geometry + color) | | Sparse basis | Global complex exponential Fourier basis $\{e^{i\mathbf{k}\cdot\mathbf{x}}\}$ | Local anisotropic Gaussians $\{G(\mathbf{x};\boldsymbol{\mu}_i,\boldsymbol{\Sigma}_i)\}$ | | Sparsity structure | Energy concentrated in low-frequency modes ( $k \le k_{\max}$ ) | Mass concentrated in thin surface layers | | Baseline discretization | $$1024^3$$ dense grid points | Multi-gigabyte dense voxel grids | | CS effect | Dozens of frequency modes summarize global vortex dynamics | Millions of Gaussians replace trillions of empty voxels |

FNO sparsifies globally in frequency; 3DGS sparsifies locally in space—opposite paths to extreme compression.

3. Measurement matrices and incoherent projection

The measurement $\mathbf{y} = \mathbf{\Phi}\mathbf{x} = \mathbf{\Phi}\mathbf{\Psi}\boldsymbol{\theta}$ appears as a duality:

FNO: sampling on a coarse 64×64 grid (spatial impulse measurements, incoherent with the frequency basis) enables zero-shot generalization to 1024×1024 resolution.
3DGS: multi-view 2D photographs are line-integral projections; by the Fourier slice theorem these are central-slice measurements of the 3D spectrum, allowing a dozen views to recover millions of Gaussians' positions and covariances, then rendering new viewpoints at 100+ FPS.

4. Recovery algorithms

The CS objective $\min_\theta \|\theta\|_1$ s.t. $\|\mathbf{y}-\mathbf{\Phi}\mathbf{\Psi}\theta\|_2 \le \epsilon$ is realized as:

1. FNO: operator-ized solving—complex convolutions + nonlinear activations amortize iterative optimization into a single forward pass, reducing PDE solve time to milliseconds. 2. 3DGS: Adaptive Density Control—gradient-driven cloning, splitting, and pruning of Gaussians, mathematically analogous to orthogonal matching pursuit's support-set recovery.

5. Future fusion directions

Gaussian Neural Operator (GNO): use the spatially local Gaussian basis to eliminate Gibbs oscillations FNO exhibits on irregular surfaces and local shocks.
Physics-constrained 4D Gaussians: drive 3D Gaussian temporal evolution with FNO-solved Navier-Stokes low-frequency dynamics, enabling physically faithful 100+ FPS interactive simulation.

References

1. Candès, E. J., Romberg, J., & Tao, T. (2006). *Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information*. IEEE Transactions on Information Theory, 52(2), 489-509. 2. Donoho, D. L. (2006). *Compressed sensing*. IEEE Transactions on Information Theory, 52(4), 1289-1306. 3. Li, Z., et al. (2021). *Fourier Neural Operator for Parametric Partial Differential Equations*. ICLR 2021 (Oral). arXiv:2010.08895. 4. Kerbl, B., et al. (2023). *3D Gaussian Splatting for Real-Time Radiance Field Rendering*. ACM Transactions on Graphics (SIGGRAPH 2023), 42(4), 139.