DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Forum topic · 小凯 · 2026-10-04

Summary

DMAD (Distribution Matching as Adversarial Distillation) is a new method for distilling diffusion models into few-step generators, presented in arXiv paper 2610.02188. Unlike Distribution Matching Distillation (DMD), which requires maintaining an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and compute cost, DMAD recasts distribution matching as a classification problem and learns log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from student samples, and linear losses on their logits train the student without auxiliary score fitting. The authors prove that at the discriminator optimum these losses recover DMD's distribution-matching gradient via the classical discriminator-logit/log-density-ratio identity. A gap-based reweighting scheme adaptively adjusts teacher supervision across noise levels. DMAD achieves FID 1.04 on ImageNet-64x64 one-step generation, FID 14.47 with SDXL four-step on COCO-10K, and VBench 85.15 with Wan2.1-T2V-14B four-step, outperforming compared few-step methods. On MiniMax-H3-33B, the four-step student achieves 79.1% human preference over DMD2 and 84.6% over rCM.

Overview

Field: Computer Vision Authors: Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma Published: 2026-10-01 arXiv: 2610.02188

Key Idea

Distribution Matching Distillation (DMD) trains a few-step student model from the difference between separately estimated target and student scores, which requires an auxiliary diffusion model fitted to the student's evolving distribution — incurring extra memory and computation cost. DMAD (Distribution Matching as Adversarial Distillation) removes this need by recasting distribution matching as a classification problem, learning the required log-density ratios directly.

Two discriminator heads on a shared backbone distinguish real data and teacher samples from student samples.
Linear losses on their logits train the student without auxiliary score fitting.
At the discriminator optimum, these losses provably recover the distribution-matching gradient underlying DMD, via the classical identity linking discriminator logits to log-density ratios.
A gap-based reweighting scheme uses the empirical logit gap on the real-data head between real and teacher samples to adaptively adjust teacher supervision strength across noise levels.

Results

ImageNet-64x64, one-step generation: FID 1.04
SDXL, four steps on COCO-10K: FID 14.47
Wan2.1-T2V-14B, four steps: VBench total score 85.15

All are the best among compared few-step methods and multi-step teachers.

On MiniMax-H3-33B, the four-step student achieves an overall human preference rate of 79.1% over DMD2 and 84.6% over rCM (excluding ties, joint audio-video generation).