Weighted Universal Approximation of Differentiable Maps on Infinite-Dimensional Manifolds

Summary

Philipp Schmocker and Josef Teichmann (arXiv:2506.04839) extend the universal approximation theorem for functional input neural networks (FNNs) to differentiable maps, including approximation of derivatives. An FNN maps inputs from a possibly infinite-dimensional weighted manifold to a real-valued hidden layer, applies a non-linear scalar activation function, and returns output into a Banach space via linear readouts. The key contribution is a proof of a weighted Nachbin theorem, from which the authors derive a universal approximation theorem for differentiable maps that goes beyond the usual compact-set formulation and covers derivative approximation as well. This yields approximation results for non-anticipative functionals, including their horizontal and vertical derivatives. As a further application, the authors show that linear functionals of the signature can approximate path-space functionals and their directional derivatives. The work is relevant to machine learning theory, rough path analysis, and deep learning on infinite-dimensional function spaces.