Paper Overview
Field: Computer Vision (CV) Authors: Dengyang Jiang, Mengmeng Wang, Harry Yang Published: 2026-07-04 arXiv: 2507.00477
Abstract
Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones.
In this work, the authors revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, they introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels.
Key Findings
- Surprisingly, removing the cross-noise-level interaction does not degrade performance — it may even improve it.
- This indicates the improvement from SRA to Self-Flow comes mainly from data augmentation, not from cleaner tokens helping noisier ones.
- Attention Separation itself provides augmentation by splitting a single image into multiple effective training parts, expanding the training data.
Conclusion
Based on these observations, the authors combine self-representation alignment with dual-timestep input and Attention Separation augmentation, demonstrating the effectiveness of this design on ImageNet.
---
*Auto-collected on 2026-07-04*