DemoPSD: Disagreement-Modulated Policy Self-Distillation

Forum topic · 小凯 · 2026-08-28

Summary

DemoPSD is a novel framework in machine learning (cs.LG, cs.AI) that addresses the problem of privileged information leakage during knowledge distillation. Instead of blindly copying teacher outputs, DemoPSD selectively adopts teacher guidance and steers the student policy toward a reverse-KL barycenter target. This balancing mechanism allows the student model to benefit from teacher knowledge while preserving its own reasoning capacity, mitigating degradation caused by over-reliance on privileged signals. The paper was authored by Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, and Linqi Song, and released on arXiv on July 2, 2026 (arXiv:2607.02502). The approach is relevant to researchers working on policy distillation, reinforcement learning, and teacher-student training paradigms where handling asymmetric information between teacher and student is a key challenge.

Paper Overview

Research areas: cs.LG, cs.AI

Authors: Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, Linqi Song

Release date: 2026-07-02

arXiv: 2607.02502

Abstract

We introduce DemoPSD, a novel framework that resolves privileged information leakage through selective adoption of teacher guidance. DemoPSD steers the student toward a reverse-KL barycenter target that balances learning from the teacher with preserving the student's own reasoning capacity.

Key Takeaways

Problem addressed: Privileged information leakage — students may over-rely on teacher signals derived from information unavailable at inference time.
Proposed solution: Selective adoption of teacher guidance via disagreement modulation, rather than unconditional imitation.
Core mechanism: The student is guided toward a reverse-KL barycenter target, balancing teacher learning with retention of the student's own reasoning capacity.
Relevance: Applicable to policy distillation and teacher-student training setups where teacher and student have asymmetric information access.