Geometric Regularization of Autoencoders via Observed Stochastic Dynamics: Tangent-Bundle Penalties for Latent SDEs

Forum topic · 小凯 · 2026-04-21

Summary

A 2026 arXiv paper (2604.16282) by Sean Hill and Felix X.-F. Ye addresses reduced simulation of stochastic dynamical systems with slow or metastable behavior that evolve on an unknown low-dimensional manifold in high-dimensional ambient space. The authors observe that ambient covariance already encodes coordinate-invariant tangent-space information, with its range spanning the tangent bundle. Leveraging this, they introduce tangent-bundle and inverse-consistency penalties for a three-stage pipeline of chart learning, latent drift, and latent diffusion, yielding a single nonlinear chart plus a latent SDE. Theoretically, the penalties induce a function-space metric strictly weaker than the Sobolev W1,1 norm while matching chart-quality generalization up to log factors; an encoder-pullback drift objective derived via Itô's formula comes with a bias decomposition showing systematic error in decoder-side formulations, with W2,infty chart convergence controlling propagation to weak convergence and mean first-passage times. Experiments on four manifolds with ambient dimensions up to 201 show 50-70% reduction in radial MFPT error under rotating dynamics and lowest inter-well MFPT error on metastable Müller-Brown Langevin dynamics, cutting end-to-end ambient coefficient error by up to an order of magnitude versus unregularized autoencoders.

Paper Overview

Field: Machine Learning Authors: Sean Hill, Felix X.-F. Ye Posted: 2026-04-17 arXiv: 2604.16282

Abstract (translated/condensed)

Stochastic dynamical systems with slow or metastable behavior evolve, on long time scales, on an unknown low-dimensional manifold in high-dimensional ambient space. Building a reduced simulator from short-burst ambient ensembles is a long-standing problem: local-chart methods like ATLAS suffer from exponential landmark scaling and per-step reprojection, while autoencoder alternatives leave tangent-bundle geometry poorly constrained, and the errors propagate into the learned drift and diffusion.

The key observation is that the ambient covariance $\Lambda$ already encodes coordinate-invariant tangent-space information, its range spanning the tangent bundle. Using this, the authors construct a tangent-bundle penalty and an inverse-consistency penalty for a three-stage pipeline (chart learning, latent drift, latent diffusion), learning a single nonlinear chart and a latent SDE.

Theoretical Contributions

The penalties induce a function-space metric strictly weaker than the Sobolev $W^{1,1}$ norm, yet achieving the same chart-quality generalization rate (up to log factors).
For drift learning, an encoder-pullback objective is derived on the learned encoder via Itô's formula, together with a bias decomposition showing that the standard decoder-side formulation carries systematic error for any imperfect chart.
Under a $W^{2,\infty}$ chart-convergence assumption, chart-level error provably propagates to weak convergence of the ambient dynamics and radial-averaged first-passage times.

Experimental Results

Four manifolds with ambient dimensions up to 201:

50–70% reduction in radial mean-first-passage-time (MFPT) error under rotating dynamics.
Lowest inter-well MFPT error on metastable Müller-Brown Langevin dynamics across most manifold-transition pairs.
End-to-end ambient coefficient error reduced by up to an order of magnitude compared to unregularized autoencoders.

---

*Auto-collected on 2026-04-21*