ShotStream: Streaming Causal Multi-Shot Video Generation for Interactive Storytelling

Forum topic · 小凯 · 2026-03-28

Summary

ShotStream (arXiv: 2603.25746) is a new causal multi-shot architecture for video generation that enables interactive storytelling with low latency. The method reformulates multi-shot video generation as next-shot generation conditioned on historical context, allowing users to steer an ongoing narrative via streaming prompts. The authors fine-tune a text-to-video model into a bidirectional next-shot generator, then distill it into a causal student using Distribution Matching Distillation. Two innovations address inter-shot consistency and error accumulation: a dual-cache memory mechanism (a global context cache for cross-shot consistency and a local context cache for in-shot consistency, disambiguated with a RoPE discontinuity indicator) and a two-stage distillation strategy that progresses from in-shot self-forcing with ground-truth history to cross-shot self-forcing with self-generated history. Experiments show ShotStream generates coherent multi-shot videos with under one second latency, achieving 16 FPS on a single GPU while matching or exceeding the quality of slower bidirectional models.

Paper Overview

Field: Computer Vision Authors: Yawen Luo, Xiaoyu Shi, Junhao Zhuang, Yutian Chen, Quande Liu, Xintao Wang, Pengfei Wan, Tianfan Xue Published: 2026-03-26 arXiv: 2603.25746

Introduction

Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. ShotStream is a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts.

Method

The approach first fine-tunes a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, the authors introduce two key innovations:

1. Dual-cache memory mechanism: Maintains visual coherence. A global context cache preserves conditional frames for cross-shot consistency, while a local context cache stores generated frames within the current shot for in-shot consistency. A RoPE discontinuity indicator explicitly distinguishes the two caches to remove ambiguity.

2. Two-stage distillation strategy: Conditions first on in-shot self-forcing with ground-truth historical shots, then progressively extends to cross-shot self-forcing with self-generated history, effectively bridging the train–test gap.

Results

Extensive experiments show that ShotStream generates coherent multi-shot videos with latency under one second, reaching 16 FPS on a single GPU. It matches or exceeds the quality of slower bidirectional models, paving the way for real-time interactive storytelling.

Original Abstract (excerpt)

> Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two...

*Auto-collected on 2026-03-28.*