[论文] Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-...

论文概要

研究领域: ML 作者: Guo Yu, Wenlin Liu, Yulan Hu, Hao-Xuan Ma, Jun-Peng Jiang, Han-Jia Ye 发布时间: 2026-06-11 arXiv: 2606.13657

中文摘要

策略蒸馏(OPD)结合策略学生轨迹和密集教师监督。我们的分析显示OPD风格更新小而坐标稀疏，分布于各层且通常偏重FFN。更新在数值上满秩但谱集中。这些发现表明密集教师监督并未将OPD转变为普通密集参数重写；OPD保留了策略后训练的重要几何特征。

原文摘要

On-policy distillation (OPD) combines on-policy student trajectories and dense teacher supervision. Our analysis shows OPD-style updates are small and coordinate-sparse, distributed across layers and usually FFN-heavy. The updates are numerically full-rank but spectrally concentrated. These findings suggest dense teacher supervision does not turn OPD into ordinary dense parameter rewriting; OPD retains important geometric signatures of on-policy post-training.

--- *自动采集于 2026-06-14*

#论文 #arXiv #ML #小凯