Constructive Bounds for the Topological Expressivity of Neural Networks (arXiv:2610.13931)

Forum topic · 小凯 · 2026-10-13

Summary

A forum post introducing the paper "Constructive Bounds for the Topological Expressivity of Neural Networks" by Kosio Beshkov (arXiv:2610.13931). The paper studies neural network expressivity through the Betti numbers of network outputs β(Φ(X)) rather than the traditional decision-boundary measure β(Φ⁻¹(0)), which is better suited to regression and other tasks without decision boundaries. The author formally proves that a ReLU network with architecture (d, Σβₖ(3k+2), Σβₖ(k+1)) can express spaces homotopy equivalent to wedge sums of spheres with arbitrary Betti numbers, yielding upper bounds for any connected (β₀=1) Betti number sequence. A tighter bound is constructed via spaces homotopy equivalent to products of N spheres, giving (Σdₙ, 3Σdₙ+2N, Σdₙ+N) for Betti numbers Π(1+x^{dₙ}). The paper concludes with a conjecture of a tighter structural bound (d, d+1, d+1) for spaces homeomorphic to spheres. Posted on zhichai.net on 2026-10-13.

Paper Overview

Field: Machine Learning
Author: Kosio Beshkov
Published: 2026-10-09
arXiv: 2610.13931

Abstract (English, from source)

The expressivity of a neural network $$Φ$$ describes the class of functions that a network can express. The term topological expressivity refers to a family of measures particularly suited to understand binary classification and is often formulated in terms of the Betti numbers of a decision boundary $β(Φ^{-1}(0))$ . Despite its popularity, this measure is not as useful for regression and other problems that do not involve a decision boundary. Instead, we propose to study topological expressivity in terms of the Betti numbers of the output of a neural network $$β(Φ(X))$$ .

Key Results

A ReLU network with architecture $(d, \sum_{k=1}^{d} β_k(3k+2), \sum_{k=1}^{d} β_k(k+1))$ can express spaces homotopy equivalent to wedge sums of spheres with arbitrary Betti numbers. This provides an upper bound for any connected (β₀ = 1) sequence of Betti numbers.
By constructing spaces homotopy equivalent to products of N spheres, the paper obtains a tighter bound of $(\sum d_n, 3\sum d_n + 2N, \sum d_n + N)$ for spaces with Betti numbers $\prod (1 + x^{d_n})$ .
The paper conjectures a stronger, tight bound of $$(d, d+1, d+1)$$ on architecture structure for spaces homeomorphic to spheres.

Why It Matters

Framing topological expressivity via output-space Betti numbers extends the measure beyond binary classification to regression and other tasks, and the constructive nature of the bounds shows explicitly how ReLU architectures can realize prescribed topological complexity.

---

*Auto-collected on 2026-10-13.*