[论文] On the (non-)closedness of the set of ReLU networks

论文概要

研究领域: ML
作者: Eloi Martinet, Leon Bungert
发布时间: 2026-11-06
arXiv: 2611.09887

中文摘要

本文证明，具有单个隐藏层且神经元数不超过 N 的 ReLU 网络集合 𝒩_N 在 W^{1,p}(Ω) 中是弱闭的（Ω 为有界连通开集）。因此，该空间上每个强制性、弱下半连续的能量泛函都在 𝒩_N 上达到最小值。这涵盖了凸被积函数的 Deep Ritz 方法、Sobolev 训练和 Lipschitz 学习。此外，当权重衰减参数趋于零时，正则化问题的极小化子（至多子序列）收敛到参数范数最小的极小化子。作者还证明，具有两个有界宽度隐藏层的 ReLU 网络集合在 W^{1,p}(Ω) 和 C(Ω̅) 中都不是闭的。这否定了 Petersen、Raslan 和 Voigtlaender 的一个猜想。

原文摘要

We prove that the set \(\mathcal{N}_N\) of ReLU networks with one hidden layer and at most \(N\in\mathbb N\) neurons is weakly closed in \(W^{1,p}(Ω)\), for a bounded connected open set \(Ω\subset\mathbb R^d\). Consequently, every coercive, weakly lower semicontinuous energy on this space attains its minimum over \(\mathcal{N}_N\). This covers the Deep Ritz method with convex integrands, Sobolev training and Lipschitz learning. Moreover, as the weight decay parameter tends to zero, minimizers of the regularized problem converge, up to subsequences, to minimizers with minimal parameter norm. We also prove that the set of ReLU networks with two hidden layers of bounded widths is not closed in \(W^{1,p}(Ω)\) or in \(C(\overlineΩ)\). This disproves a conjecture of Petersen, Raslan and Voigtlaender.

自动采集于 2026-11-10

#论文 #arXiv #ML #小凯