English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

What Kills v-prediction? A Patch-wise PCA Perspective on Pixel-Space Diffusion (RiT)

Forum topic · 小凯 · 2026-11-10

Summary

This paper investigates why v-prediction often fails in pixel-space flow matching models, offering a new explanation based on patch-wise PCA analysis. While latent diffusion models dominate generation, they suffer from two-stage training complexity and reconstruction limits of pretrained autoencoders. Pixel-space flow matching is a simpler end-to-end alternative, but standard v-prediction targets frequently encounter training difficulties, recently attributed to off-manifold noise targets. The authors identify a bandwidth bottleneck instead: minor principal components of the velocity are largely determined by the raw noise input, forcing deep backbones to retain information competing with dominant-component modeling. They analyze why x-prediction still works—its parameterization provides a raw-input bypass and its representations spontaneously focus on principal directions—and characterize it as an implicit conditional mean decoder under low-rank observations. Building on these insights, they propose the Raw-injected Transformer (RiT), which separates semantic modeling from raw-state propagation via a lightweight raw-conditioned decoder and an output-side shortcut, restoring effective v-prediction and achieving competitive performance on ImageNet 256x256.

Paper Overview

  • Field: Computer Vision (CV)
  • Authors: Tianze Luo, Haofeng Huang, Yuan Yao
  • Published: 2026-11-09
  • arXiv: 2611.12852
  • Abstract (Translated)

    Latent diffusion models dominate generative modeling but face two-stage training complexity and an inherent reconstruction ceiling imposed by pretrained autoencoders. Pixel-space flow matching offers a simplified end-to-end alternative, yet it often suffers from training difficulties when using the standard v-prediction objective. Recent work attributes this to the off-manifold nature of noisy targets, motivating a shift toward x-prediction. This paper revisits that explanation through patch-wise PCA and identifies a bandwidth bottleneck: minor components of the velocity are largely determined by the raw noise input, requiring deep backbones to retain information that competes with modeling of the principal components.

    Through controlled experiments and representation analysis, the authors further examine why x-prediction remains effective: its parameterization provides a bypass of the raw input, and its learned input representations can spontaneously concentrate on principal directions. Under low-rank observations, analysis of the induced flow characterizes x-prediction as an implicit conditional mean decoder—reconstructing discarded components from retained information.

    Based on these observations, the paper proposes the Raw-injected Transformer (RiT): a lightweight raw-conditioned decoder combined with an output-side shortcut, separating semantic modeling from raw-state propagation. This approach restores effective v-prediction and achieves competitive performance on ImageNet 256×256, offering a new perspective on the interaction between prediction objectives and architectural bandwidth.

    Original Abstract

    > We investigate training difficulties with v-prediction in pixel-space flow matching through patch-wise PCA, identifying a bandwidth bottleneck. We propose the Raw-injected Transformer (RiT) which combines a lightweight raw-conditioned decoder with an output-side shortcut, restoring effective v-prediction on ImageNet 256x256.

    Links

  • arXiv page: https://arxiv.org/abs/2611.12852

Tags

#diffusion-models#flow-matching#v-prediction#patch-wise-pca#transformer#image-generation#paper#cv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637023