[论文] DMAD: Distribution Matching as Adversarial Distillation for Fast Visua...

论文概要

研究领域: CV 作者: Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma 发布时间: 2026-10-01 arXiv: 2610.02188

中文摘要

分布匹配蒸馏（DMD）通过分别估计的目标与学生分数之差训练少步数学生模型，因此必须额外维护一个拟合学生不断变化分布的辅助扩散模型，带来额外内存与计算开销。我们提出 DMAD——分布匹配即对抗蒸馏：将分布匹配重构为分类问题，直接学习所需的对数密度比。共享骨干网络上的两个判别器头区分真实数据、教师样本与学生样本，对其 logits 施加线性损失即可训练学生，无需辅助分数拟合。我们证明：在判别器最优处，这些损失经由经典的「判别器 logit ↔ 对数密度比」恒等式，恢复了 DMD 底层的分布匹配梯度。我们还引入基于差距的重加权：利用真实数据头在真实与教师样本间的经验 logit 差距，自适应调整跨噪声水平的教师监督强度。DMAD 在 ImageNet-64x64 一步生成达 FID 1.04，SDXL 四步在 COCO-10K 上达 14.47，Wan2.1-T2V-14B 四步 VBench 总分 85.15，均为所比较少步数方法与多步教师中的最佳。在 MiniMax-H3-33B 上，四步学生对 DMD2 取得 79.1%、对 rCM 取得 84.6% 的总体人类偏好率（不含平局，音画联合生成）。代码、模型与演示见 https://yzmblog.github.io/projects/DMAD

原文摘要

Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discr...

#论文 #arXiv #CV #小凯