CalibAtt: Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Forum topic · 小凯 · 2026-03-07

Summary

This paper introduces CalibAtt, a training-free method for accelerating text-to-video diffusion models via calibrated sparse attention. The authors observe that a significant fraction of token-to-token attention connections consistently produce negligible scores across inputs, and these patterns often repeat across queries, allowing attention computation to be skipped with little to no effect on output quality. This holds for connections among local token blocks as well. CalibAtt performs an offline calibration pass that identifies block-level sparsity and repetition, then exploits these patterns to reduce the costly spatiotemporal attention bottleneck in large transformer-based video generation backbones. The work falls in the computer vision domain, addressing slow runtimes of diffusion-based video generation while preserving quality. arXiv: 2603.05503.

Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Authors: Shai Yehezkel, Shahar Yadin, Noam Elata, Yaron Ostrovsky-Berman, Bahjat Kawar

arXiv: 2603.05503
PDF: https://arxiv.org/pdf/2603.05503.pdf
Category: cs.CV (Computer Vision)

Abstract

Recent diffusion models enable high-quality video generation, but suffer from slow runtimes. The large transformer-based backbones used in these models are bottlenecked by spatiotemporal attention. In this paper, the authors identify that a significant fraction of token-to-token connections consistently yield negligible scores across various inputs, and their patterns often repeat across queries. Thus, the attention computation in these cases can be skipped with little to no effect on the result. This observation continues to hold for connections among local token blocks.

Motivated by this, the paper introduces CalibAtt, a training-free method that accelerates video generation via calibrated sparse attention. CalibAtt performs an offline calibration pass that identifies block-level sparsity and repetition in attention patterns, then uses these findings to skip redundant attention computation during inference.

Key Points

Method: Transformer, Attention, Diffusion
Research type: Empirical study in computer vision
Problem addressed: Slow runtimes of diffusion-based video generation caused by spatiotemporal attention bottlenecks
Approach: Training-free sparse attention with an offline calibration pass to detect block-level sparsity and repeated attention patterns

*(Automatically collected on 2026-03-07)*