Paper Overview
- Field: Computer Vision (CV)
- Authors: Tianze Luo, Haofeng Huang, Yuan Yao
- Published: 2026-11-09
- arXiv: 2611.12852
- arXiv page: https://arxiv.org/abs/2611.12852
Abstract (Translated)
Latent diffusion models dominate generative modeling but face two-stage training complexity and an inherent reconstruction ceiling imposed by pretrained autoencoders. Pixel-space flow matching offers a simplified end-to-end alternative, yet it often suffers from training difficulties when using the standard v-prediction objective. Recent work attributes this to the off-manifold nature of noisy targets, motivating a shift toward x-prediction. This paper revisits that explanation through patch-wise PCA and identifies a bandwidth bottleneck: minor components of the velocity are largely determined by the raw noise input, requiring deep backbones to retain information that competes with modeling of the principal components.
Through controlled experiments and representation analysis, the authors further examine why x-prediction remains effective: its parameterization provides a bypass of the raw input, and its learned input representations can spontaneously concentrate on principal directions. Under low-rank observations, analysis of the induced flow characterizes x-prediction as an implicit conditional mean decoder—reconstructing discarded components from retained information.
Based on these observations, the paper proposes the Raw-injected Transformer (RiT): a lightweight raw-conditioned decoder combined with an output-side shortcut, separating semantic modeling from raw-state propagation. This approach restores effective v-prediction and achieves competitive performance on ImageNet 256×256, offering a new perspective on the interaction between prediction objectives and architectural bandwidth.
Original Abstract
> We investigate training difficulties with v-prediction in pixel-space flow matching through patch-wise PCA, identifying a bandwidth bottleneck. We propose the Raw-injected Transformer (RiT) which combines a lightweight raw-conditioned decoder with an output-side shortcut, restoring effective v-prediction on ImageNet 256x256.