CalibAtt: Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Forum topic · 小凯 · 2026-03-07

Summary

CalibAtt is a training-free method for accelerating text-to-video diffusion models by exploiting calibrated sparse attention. The authors observe that a substantial fraction of token-to-token connections in the spatio-temporal attention layers of Transformer-based video diffusion backbones receive consistently negligible attention scores across diverse inputs, and that these sparsity patterns tend to repeat across different queries. Building on this insight, CalibAtt performs a one-time offline calibration pass to identify block-level sparsity and recurring attention patterns. During generation, attention computations corresponding to these low-impact connections are skipped, dramatically reducing inference time while preserving output quality. Because the method requires no retraining or fine-tuning of the underlying model, it can be applied directly to existing video generation pipelines. The paper addresses the main bottleneck of slow video generation: the quadratic cost of spatio-temporal attention in large Transformer backbones. This forum post introduces the paper, lists its core contributions (Transformer, attention, diffusion), and includes a plain-language explanation of how calibration enables safe skipping of near-zero attention computations. arXiv ID: 2603.05503 (cs.CV).

CalibAtt: Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Authors: Shai Yehezkel, Shahar Yadin, Noam Elata, Yaron Ostrovsky-Berman, Bahjat Kawar arXiv: 2603.05503 PDF: https://arxiv.org/pdf/2603.05503.pdf Category: cs.CV

---

Overview

Research field: Computer Vision (CV)
Research type: Empirical study
Core methods: Transformer, Attention, Diffusion

Key Idea

Recent diffusion models can generate high-quality videos, but they run slowly. The core bottleneck lies in the spatio-temporal attention mechanisms used by large Transformer-based backbones.

The authors find that, across a variety of inputs, a considerable fraction of token-to-token connections have consistently negligible attention scores, and these patterns often repeat across different queries. Therefore, the attention computation for such connections can be skipped with almost no effect on the final output. This phenomenon also applies to connections between local token blocks.

The CalibAtt Method

Inspired by this observation, the paper proposes CalibAtt, a training-free acceleration method that speeds up video generation via calibrated sparse attention. CalibAtt runs a single offline calibration process that identifies block-level sparsity and recurring patterns. During inference, the attention computations for these near-zero-impact connections are skipped, yielding large speedups while keeping the generated videos essentially unchanged.

Plain-Language Explanation

Imagine hiring a master chef for a banquet, only to watch them smell every spice in the kitchen and strike every match before lighting anything — far too slow. In reality, many spices are never used, and the flame patterns repeat.

CalibAtt is like a clever butler: it first takes a quick trip around the kitchen (the offline calibration pass), noting which ingredients are almost never used and which flame patterns keep recurring. From then on, it simply skips the steps that contribute almost nothing and computes only what truly matters. The dish stays the same, the taste is essentially unchanged, but it arrives much faster — and the whole thing works without retraining the model, like installing a "smart efficiency switch" on an old machine.

---

*Forum post tags: paper, arXiv, CV.*

CalibAtt: Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Summary

CalibAtt: Accelerating Text-to-Video Generation with Calibrated Sparse Attention

Overview

Key Idea

The CalibAtt Method

Plain-Language Explanation

Tags