From SRA to Self-Flow: Data Augmentation or Self-Supervision? (arXiv 2507.00477)

Forum topic · 小凯 · 2026-07-04

Summary

This paper (arXiv:2507.00477) by Dengyang Jiang, Mengmeng Wang, and Harry Yang examines the mechanism behind the improvement from SRA to Self-Flow in self-alignment methods for diffusion transformers. Representation alignment accelerates diffusion transformer training and improves generation quality, and recent self-alignment approaches like SRA and Self-Flow remove reliance on external pretrained encoders. Self-Flow attributes its gains to dual-time scheduling, where interactions between tokens at different noise levels let cleaner tokens help infer noisier ones. The authors challenge this explanation, asking whether the gains instead come from data augmentation along the noise dimension. They introduce Attention Separation, which keeps the same dual-timestep input but blocks attention between tokens assigned to different noise levels. Surprisingly, removing this interaction does not degrade performance and may even improve it, suggesting the SRA-to-Self-Flow improvement primarily comes from data augmentation. They further show that Attention Separation itself augments training data by splitting a single image into multiple effective training parts. Combining self-representation alignment with dual-timestep inputs and Attention Separation augmentation, they demonstrate the design's effectiveness on ImageNet.

Paper Overview

Field: Computer Vision (CV) Authors: Dengyang Jiang, Mengmeng Wang, Harry Yang Published: 2026-07-04 arXiv: 2507.00477

English Abstract (from the paper)

Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension.

Key Points

Motivation: The mechanism behind SRA → Self-Flow improvement (dual-time scheduling) was under-examined; the paper questions whether cross-noise-level token interaction is the true source of the gains.
Method: To disentangle the factors, the authors introduce Attention Separation, which preserves the same dual-timestep input while blocking attention between tokens assigned to different noise levels.
Surprising finding: Removing the cross-noise-level interaction does not hurt performance and may even improve it — suggesting the improvement from SRA to Self-Flow mainly comes from data augmentation, not self-supervision via token interaction.
Augmentation effect: Attention Separation itself expands training data by splitting a single image into multiple effective training parts.
Result: Combining self-representation alignment with dual-timestep inputs and Attention Separation augmentation, the authors demonstrate the design's effectiveness on ImageNet.

---

*Auto-collected on 2026-07-04*