Paper: Dense Supervision, Sparse Updates — The Sparsity and Geometry of On-Policy Distillation

Forum topic · 小凯 · 2026-06-14

Summary

A forum post introduces an arXiv paper (2606.13657) by Guo Yu, Wenlin Liu, Yulan Hu, Hao-Xuan Ma, Jun-Peng Jiang, and Han-Jia Ye analyzing on-policy distillation (OPD), a technique that combines on-policy student trajectories with dense teacher supervision. The authors' analysis shows that OPD-style parameter updates are small in magnitude and coordinate-sparse, yet distributed across model layers and typically concentrated in feed-forward network (FFN) components. Although the updates are numerically full-rank, their spectrum is concentrated, indicating low effective dimensionality. These findings suggest that dense teacher supervision does not transform OPD into ordinary dense parameter rewriting; instead, OPD retains key geometric signatures of on-policy post-training. The post includes the paper metadata, arXiv link, and both a Chinese and original English abstract, and was automatically collected on 2026-06-14.

Paper Overview

Research Area: ML Authors: Guo Yu, Wenlin Liu, Yulan Hu, Hao-Xuan Ma, Jun-Peng Jiang, Han-Jia Ye Published: 2026-06-11 arXiv: 2606.13657

Abstract

On-policy distillation (OPD) combines on-policy student trajectories and dense teacher supervision. Our analysis shows OPD-style updates are small and coordinate-sparse, distributed across layers and usually FFN-heavy. The updates are numerically full-rank but spectrally concentrated. These findings suggest dense teacher supervision does not turn OPD into ordinary dense parameter rewriting; OPD retains important geometric signatures of on-policy post-training.

Key Points

OPD-style updates are small in magnitude and coordinate-sparse.
Updates are distributed across layers and tend to be FFN-heavy.
The update matrices are numerically full-rank but spectrally concentrated (low effective dimensionality).
Conclusion: dense teacher supervision does not reduce OPD to ordinary dense parameter rewriting — OPD preserves important geometric signatures of on-policy post-training.

---

*Auto-collected on 2026-06-14.*