Summary
CalibAtt is a training-free method that accelerates text-to-video diffusion models through calibrated sparse attention. The authors observe that a significant fraction of token-to-token connections in the spatiotemporal attention layers of transformer-based video diffusion backbones consistently yield negligible attention scores across diverse inputs, and that these patterns often repeat across queries. This means attention computations can be safely skipped with little to no degradation in output quality. Building on this observation, CalibAtt performs an offline calibration pass that identifies block-level sparsity and repetition, then exploits it during video generation to skip redundant attention computation. The approach addresses the slow runtime bottleneck caused by spatiotemporal attention in large video generation models without requiring retraining. The work falls under computer vision and empirical research, touching on Transformers, attention mechanisms, and diffusion models.
Accelerating Text-to-Video Generation with Calibrated Sparse Attention
Authors: Shai Yehezkel, Shahar Yadin, Noam Elata, Yaron Ostrovsky-Berman, Bahjat Kawar
arXiv: 2603.05503
PDF: https://arxiv.org/pdf/2603.05503.pdf
Category: cs.CV (Computer Vision)
Key Points
- Problem: Recent diffusion models enable high-quality video generation, but suffer from slow runtimes. The large transformer-based backbones used in these models are bottlenecked by spatiotemporal attention.
- Observation: A significant fraction of token-to-token connections consistently yield negligible attention scores across various inputs, and their patterns often repeat across queries. Attention computation in these cases can be skipped with little to no effect on the result. This holds for connections among local token blocks as well.
- Method: CalibAtt, a training-free acceleration method based on calibrated sparse attention. It performs an offline calibration pass that identifies block-level sparsity and repetition, which is then exploited to skip redundant attention computation during generation.
> Recent diffusion models enable high-quality video generation, but suffer from slow runtimes. The large transformer-based backbones used in these models are bottlenecked by spatiotemporal attention. In this paper, we identify that a significant fraction of token-to-token connections consistently yield negligible scores across various inputs, and their patterns often repeat across queries. Thus, the attention computation in these cases can be skipped with little to no effect on the result. This observation continues to hold for connections among local token blocks. Motivated by this, we introduce CalibAtt, a training-free method that accelerates video generation via calibrated sparse attention. CalibAtt performs an offline calibration pass that identifies block-level sparsity and repetition ...
*Auto-collected on 2026-03-07. Note: the source abstract is truncated; see the arXiv link for the full paper.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177168725