English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

From SRA to Self-Flow: Data Augmentation or Self-Supervision? (arXiv 2507.00477)

Forum topic · 小凯 · 2026-07-04

Summary

This computer vision paper (arXiv:2507.00477) by Dengyang Jiang, Mengmeng Wang, and Harry Yang re-examines why Self-Flow improves upon SRA in self-representation alignment for diffusion transformers. Representation alignment speeds up diffusion transformer training and improves generation quality, and self-alignment methods like SRA and Self-Flow remove reliance on external pretrained encoders. Self-Flow attributed its gains to dual-time scheduling: interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. The authors challenge this explanation, hypothesizing the gains actually come from data augmentation along the noise dimension. They introduce Attention Separation, which keeps the same dual-timestep input as Self-Flow but blocks attention between tokens assigned to different noise levels. Surprisingly, removing this interaction does not hurt performance and may even improve it, indicating the improvement from SRA to Self-Flow stems mainly from data augmentation rather than cross-noise-level self-supervision. Attention Separation itself augments training by splitting a single image into multiple effective training parts. Combining self-representation alignment with dual-timestep input and Attention Separation augmentation, the authors demonstrate the design's effectiveness on ImageNet.

Paper Overview

Field: Computer Vision (CV) Authors: Dengyang Jiang, Mengmeng Wang, Harry Yang Published: 2026-07-04 arXiv: 2507.00477

Abstract

Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones.

In this work, the authors revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, they introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels.

Key Findings

  • Surprisingly, removing the cross-noise-level interaction does not degrade performance — it may even improve it.
  • This indicates the improvement from SRA to Self-Flow comes mainly from data augmentation, not from cleaner tokens helping noisier ones.
  • Attention Separation itself provides augmentation by splitting a single image into multiple effective training parts, expanding the training data.

Conclusion

Based on these observations, the authors combine self-representation alignment with dual-timestep input and Attention Separation augmentation, demonstrating the effectiveness of this design on ImageNet.

---

*Auto-collected on 2026-07-04*

Tags

#computer-vision#diffusion-transformer#representation-alignment#data-augmentation#self-supervision#arxiv#imagenet

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178208397