[论文] Learning between the peaks: sharp asymptotics for kernel ridge regress...

论文概要

研究领域: ML 作者: Lorenzo Rizzi, Arie Wortsman Zurich, Bruno Loureiro 发布时间: 2026-08-28 arXiv: 2608.28564

中文摘要

我们在各向异性高斯数据下研究核岭回归，其中输入协方差以指数α≥0的幂律衰减。我们在多项式高维情形n=Θ(d^κ)下推导了核谱和泛化误差的渐近精确表达式，揭示了各向异性如何重塑学习曲线。对于弱各向异性（0<α<1），问题保持有效高维并保留各向同性情形的一些特征，但在其他方面偏离：方差仍在整数样本复杂度κ∈ℕ处达到峰值，但随着α增大这些峰值逐渐被抑制；同时，对于与数据主方向强对齐的目标，偏差在分数样本复杂度处下降，使偏差过渡与插值峰值解耦。对于强各向异性（α>1），问题的有效维度恒定，方差完全不再依赖于样本量，在无岭插值下趋于平稳或在固定岭惩罚下以显式速率消失。偏差经历由目标衰减率控制的急剧转变：低于阈值时，学习是突变而非渐进的；高于阈值时，偏差以幂律衰减，恢复经典的源和容量速率。我们最终将这些结果专门用于单指标目标，展示指标与数据主方向的对齐如何决定各向异性对学习的影响。

原文摘要

We study kernel ridge regression under anisotropic Gaussian data, where the input covariance decays as a power law with exponent \(α\geq 0\) for polynomial inner-product kernels. We derive asymptotically sharp expressions for the kernel spectrum and the generalization error in the polynomial high-dimensional regime \(n=Θ(d^κ)\), revealing how anisotropy reshapes the learning curves. For weak anisotropy (\(0<α<1\)), the problem remains effectively high-dimensional and retains some features of the isotropic case, while departing from it in others: the variance still peaks at integer sample complexities \(κ\in\mathbb{N}\), but these peaks are progressively damped as \(α\) grows; meanwhile, for targets strongly aligned with the data's principal directions, the bias drops at fractional sample complexitie...

#论文 #arXiv #ML #小凯