CtrlCache: Accelerating Interactive Video World Models with Control-Aware Caching

Forum topic · 小凯 · 2026-10-08

Summary

CtrlCache is a training-free, control-aware caching framework that accelerates interactive video world models. Interactive systems typically use chunk-wise autoregressive generation with few-step denoising, but each chunk still requires several expensive denoising iterations. Existing caching policies base reuse decisions on model-internal denoising dynamics, ignoring a free signal: the control inputs for a chunk arrive before denoising, so scheduling based on them costs no forward pass. The authors show that structural similarity between adjacent chunks drops around action changes, while low-frequency structure persists longer than high-frequency details. CtrlCache therefore tags each chunk as initial, transitional, steering, or stable using an action-aware schedule, fully computes initial and transitional chunks, and reuses transformer residuals for steering and stable chunks. A frequency-blended historical prior guidance incorporates complementary information from previous clean latents during stable interactions without extra DiT forward passes. On Matrix-Game 2.0 and LingBot-World v1/v2, CtrlCache achieves 1.21x-1.41x DiT backbone speedup while improving WBench scores across all three models, without retraining. Paper: arXiv 2610.08777.

Paper Overview

Research area: Computer Vision (CV)
Authors: Shangye Song, Dong Gong, Hong Jia, Yun Sing Koh, Xinyu Zhang
Published: 2026-10-06
arXiv: 2610.08777

Abstract

Interactive video world models need to generate each video chunk efficiently while responding faithfully to user controls. Many systems use chunk-wise autoregressive generation with few-step denoising, but each chunk still requires several costly denoising iterations. Training-free caching can reduce this cost, yet existing policies make reuse decisions primarily from model-internal denoising dynamics and do not explicitly account for control transitions. Actually, interactive generation explicitly exposes a signal they do not use: the controls for a chunk arrive before it is denoised, so a schedule derived from them costs no forward pass.

Key Ideas

Observation: Analyzing adjacent chunks under different control regimes shows that structural similarity drops around action changes, while low-frequency structure persists longer than high-frequency details.
CtrlCache: A training-free, control-aware caching framework that adapts computation to the current control sequence.
Action-aware schedule and refresh policy: Detects action changes across and within chunks, tagging each chunk as initial, transitional, steering, or stable. At a chosen internal denoising step, initial and transitional chunks keep full computation, while steering and stable chunks reuse transformer residuals from the most recent fully computed step within the same chunk.
Frequency-blended historical prior guidance: Exploits the persistence of low-frequency structure during stable interactions, incorporating complementary information from the previous clean latent without extra DiT forward passes.

Results

Evaluated on Matrix-Game 2.0 and LingBot-World v1/v2, CtrlCache achieves 1.21x-1.41x speedup of the DiT backbone while improving WBench total scores on all three models - all without retraining the model.

*Auto-collected on 2026-10-08.*