BASA: Backend-Agnostic Sparse Attention for Fast High-Resolution Diffusion Transformers

Forum topic · 小凯 · 2026-10-08

Summary

BASA is a backend-agnostic sparse attention method for Diffusion Transformers (DiTs), addressing the quadratic cost of full attention in high-resolution image and video generation. Existing window attention faces a trade-off: partitioned windows are fast but block cross-window interaction, causing visible grid artifacts, while fine-grained sliding-window attention restores quality but has irregular computation patterns that create gaps between theoretical and practical speedup and requires custom per-hardware kernels. BASA replaces visual self-attention with shifted local window attention, introducing a structured window-shift scheme across DiT blocks so tokens separated by window boundaries communicate in later layers, enabling global information exchange and eliminating window-induced artifacts. The design adds no irregular operators or custom kernels, allowing direct deployment on existing attention backends. Experiments show BASA achieves over 90% of theoretically estimated speedup on FLUX and a 4.52x attention speedup on Wan while maintaining competitive generation quality.

This post summarizes the paper "Backend-Agnostic Sparse Attention for Fast High-Resolution Visual Generation" (computer vision), by Liao Ma, Jiayi Song, Yunfeng Wu, Songhua Liu, and Peilin Zhao, released 2026-10-06 on arXiv: 2610.08772.

Overview

Diffusion Transformers (DiTs) have achieved strong performance in image and video generation, but the quadratic complexity of full attention makes high-resolution generation computationally expensive. Window attention offers an efficient alternative, yet existing methods face a practical trade-off:

Partitioned window attention achieves computational efficiency consistent with its theoretical complexity. However, isolated windows block cross-window interaction, often introducing visible grid-like artifacts in the generated results.
Fine-grained sliding-window attention effectively restores interactions across neighboring windows and improves visual quality. However, its irregular computation patterns create a substantial gap between theoretical and practical speedups and require specialized kernels tailored to each hardware backend.

The BASA Method

BASA (Backend-Agnostic Sparse Attention) achieves both visual quality and practical acceleration:

It replaces visual self-attention with shifted local window attention.
A structured window-shift scheme across DiT blocks allows tokens separated by window boundaries to communicate in subsequent layers, enabling global information exchange and eliminating window-induced visual artifacts.
The design introduces no extra irregular operators or custom kernels, making it directly deployable on existing attention backends.

Results

On FLUX: over 90% of the theoretically estimated speedup realized in practice.
On Wan: 4.52x attention speedup.
Generation quality remains competitive with baselines.

Paper: https://arxiv.org/abs/2610.08772