Weighted Universal Approximation of Differentiable Maps on Infinite-Dimensional Manifolds

Forum topic · 小凯 · 2026-06-10

Summary

This paper by Philipp Schmocker and Josef Teichmann (arXiv:2506.04839, June 2025) generalizes the universal approximation theorem (UAT) for functional input neural networks (FNNs) to differentiable maps, including approximation of derivatives. An FNN maps inputs from a possibly infinite-dimensional weighted manifold to a real-valued hidden layer, applies a non-linear scalar activation function, and returns output to a Banach space via linear readouts. By proving a weighted Nachbin theorem, the authors establish a UAT for differentiable maps that goes beyond the usual compact-set formulation and covers derivative approximation. This yields approximation results for non-anticipative functionals, including horizontal and vertical derivatives. As a further application, they show that linear functionals of the signature can approximate path-space functionals and their directional derivatives.

Overview

Field: Machine Learning
Authors: Philipp Schmocker, Josef Teichmann
Published: 2025-06-06
arXiv: 2506.04839

Abstract (original)

We generalize the universal approximation theorem for functional input neural networks (FNN) to differentiable maps by including the approximation of the derivatives. A FNN maps the input from a possibly infinite-dimensional weighted manifold to the real-valued hidden layer, on which a non-linear scalar activation function is applied, and then returns the output into a Banach space via some linear readouts. By proving a weighted Nachbin theorem, we establish a universal approximation theorem (UAT) for differentiable maps, which goes beyond the usual formulation on compact sets and also includes the approximation of the derivatives. This leads us to approximation results for non-anticipative functionals including the horizontal and vertical derivatives. As a further application, we show that linear functions of the signature can approximate path-space functionals and their directional derivatives.

Key Contributions

Extends the UAT for FNNs from continuous functions to differentiable maps, with simultaneous approximation of derivatives.
Proves a weighted Nachbin theorem, enabling approximation beyond compact sets in possibly infinite-dimensional weighted manifolds.
Derives approximation guarantees for non-anticipative functionals, including horizontal and vertical derivatives.
Shows that linear functionals of the signature approximate path-space functionals and their directional derivatives.