Paper overview
Research area: Computer vision Authors: Tianhong Li, Huiwen Chang, Kai Zhang, et al. Release date: April 30, 2026 arXiv: 2604.28190
Abstract
We show that Fréchet Distance (FD), long considered impractical as a training objective, can in fact be effectively optimized in representation space. Our idea is simple: decouple the population size used for FD estimation, such as 50,000 samples, from the batch size used for gradient computation, such as 1,024 samples. We call this approach FD-loss.
Optimizing FD-loss reveals several surprising findings. First, post-training a base generator with FD-loss in different representation spaces consistently improves visual quality. In Inception feature space, a one-step generator reaches an FID of 0.72 on ImageNet at 256×256 resolution.
Second, the same FD-loss can transform a multi-step generator into a powerful one-step generator without teacher distillation, adversarial training, or per-sample objectives.
Third, FID may rank visual quality incorrectly. Modern representations can produce better samples even when their Inception FID is worse. This observation motivates FDr^k, a multi-representation metric that evaluates generated samples across multiple feature spaces.
We hope this work encourages further research into distribution distances in diverse representation spaces as both training objectives and evaluation metrics for generative models.