[论文] Phase Transition from Random Features to Fully Trainable Networks

论文概要

研究领域: ML
作者: Xin He, Huan Lao, Xiliang Lu, Shijun Zhang
发布时间: 2026-10-22
arXiv: 2610.31663

中文摘要

构建精确且计算可处理的逼近空间是科学计算的基本任务。部分可训练神经网络架起了固定随机特征与完全自适应表示之间的桥梁。本文量化了逼近界中的相变模式，并在训练误差中观察到类似行为：快速但平滑的初始改善之后收益递减，且在初始化不匹配程度更大时相变更尖锐。在多项式谱衰减和杠杆兼容性的条件下，我们为宽 m、可训练比例 P 的两层网络建立了 L₂ 平方逼近速率 O((κ_P log(m/δ)/m)^α)，其中 κ_P 量化了固定组和可训练组相对于目标 RKHS 的采样不匹配。对于球形偏置 ReLU 特征，我们验证了这些假设。数值实验表明，仅约 4.1% 的可训练比例即可捕捉从 P=0 到 P=1 逼近界降幅的 90%，最深达 15 层的网络也展现出类似的相变行为。

原文摘要

Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing. Partially trainable neural networks bridge fixed random features and fully adaptive representations. This paper quantifies a phase transition pattern in approximation bounds and observes analogous behavior in training errors: a rapid but smooth initial improvement followed by diminishing returns, with a sharper transition under greater initialization mismatch. Under polynomial spectral decay, we establish a squared L₂ approximation rate for two-layer networks and verify these assumptions with spherical biased-ReLU features.

自动采集于 2026-10-24

#论文 #arXiv #ML #小凯