ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling

Forum topic · 小凯 · 2026-03-28

Summary

ShotStream is a causal multi-shot video generation architecture that enables interactive storytelling with low latency. It reformulates long narrative video generation as next-shot generation conditioned on historical context, allowing users to steer ongoing narratives through streaming prompts. A text-to-video model is first fine-tuned into a bidirectional next-shot generator, then distilled into a causal student via Distribution Matching Distillation. To address inter-shot consistency and error accumulation in autoregressive generation, the authors introduce a dual-cache memory mechanism (a global context cache for cross-shot consistency and a local context cache for within-shot consistency, disambiguated by a RoPE discontinuity indicator) and a two-stage distillation strategy progressing from intra-shot self-forcing on ground-truth history to cross-shot self-forcing on self-generated history. Experiments show ShotStream generates coherent multi-shot videos with sub-second latency at 16 FPS on a single GPU, matching or exceeding slower bidirectional models. Paper: arXiv 2603.25746.

Paper Overview

Field: Computer Vision Authors: Yawen Luo, Xiaoyu Shi, Junhao Zhuang, Yutian Chen, Quande Liu, Xintao Wang, Pengfei Wan, Tianfan Xue Published: 2026-03-26 arXiv: 2603.25746

Abstract (Translated)

Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. ShotStream is a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts.

The approach first fine-tunes a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome inter-shot consistency and error accumulation inherent in autoregressive generation, the paper introduces two key innovations:

1. Dual-cache memory mechanism: maintains visual coherence — a global context cache retains conditioning frames for cross-shot consistency, while a local context cache stores frames generated within the current shot for intra-shot consistency. A RoPE discontinuity indicator explicitly disambiguates the two caches.

2. Two-stage distillation strategy: starts with intra-shot self-forcing conditioned on ground-truth historical shots, then progressively extends to cross-shot self-forcing using self-generated history, effectively bridging the train-test gap.

Key Results

Generates coherent multi-shot videos with sub-second latency, achieving 16 FPS on a single GPU
Matches or exceeds the quality of slower bidirectional models
Enables real-time interactive storytelling with streaming prompt guidance

Original Abstract (Excerpt)

> Multi-shot video generation is crucial for long narrative storytelling, yet current bidirectional architectures suffer from limited interactivity and high latency. We propose ShotStream, a novel causal multi-shot architecture that enables interactive storytelling and efficient on-the-fly frame generation. By reformulating the task as next-shot generation conditioned on historical context, ShotStream allows users to dynamically instruct ongoing narratives via streaming prompts. We achieve this by first fine-tuning a text-to-video model into a bidirectional next-shot generator, which is then distilled into a causal student via Distribution Matching Distillation. To overcome the challenges of inter-shot consistency and error accumulation inherent in autoregressive generation, we introduce two...

*Paper link: https://arxiv.org/abs/2603.25746*