Overview
Field: Machine Learning Author: Hao Chen Published: 2026-08-25 arXiv: 2608.24881
Summary
Generative models are commonly ranked by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), yet FID's first-two-moment summary can miss distributional differences, and a reported scalar gap alone is not a calibrated test against sampling variation.
FID's moment restriction has concrete consequences: on ImageNet, visually unrecognizable images optimized only to match the reference Inception mean and covariance obtain FID $24.7$ versus $58.6$ for held-out real images (lower is better).
Moreover, FID and KID are scalar discrepancies that are unchanged when the two samples are exchanged and therefore do not encode the direction of a dispersion change: under-dispersion, as can occur in mode collapse, versus over-dispersion.
ZID: Z-resolved Integrated Diagnostic
The paper introduces ZID, which combines six standardized location-sensitive and dispersion-sensitive branches derived from:
- Rank plots (RISE)
- Gaussian kernels with two bandwidths (GPK)
- FID can assign better scores to unrecognizable images than to held-out real images
- Scalar metrics like FID/KID cannot distinguish mode collapse (under-dispersion) from over-dispersion
- ZID provides calibrated testing (permutation p-values) alongside diagnostic directionality
ZID reports three linked outputs: 1. An index for ranking the magnitude of deviation 2. A permutation p-value for testing distributional equality 3. A signed dispersion reading for diagnosis
Results
In controlled experiments, ZID detects a wide range of deviations, and its scores track increasing severity across corresponding parameter scans — including cases where FID is flat or reversed.