Overview
Field: Computer Vision (CV) Authors: Boyang Wang, Guangyi Xu, Zhipeng Tang Published: 2025-04-29 arXiv: 2504.20683
Abstract
Shot Boundary Detection (SBD) aims to automatically identify shot changes and divide a video into coherent shots. While SBD was widely studied in the literature, existing state-of-the-art methods often produce non-interpretable boundaries on transitions, miss subtle yet harmful discontinuities, and rely on noisy, low-diversity annotations and outdated benchmarks.
To alleviate these limitations, the authors propose OmniShotCut, which formulates SBD as structured relational prediction, jointly estimating shot ranges with intra-shot relations and inter-shot relations via a shot query-based dense video Transformer.
Key contributions:
- Relational formulation: SBD is treated as structured relational prediction rather than independent per-frame classification, jointly modeling shot ranges, intra-shot relations, and inter-shot (cross-shot) relations.
- Shot query-based architecture: A dense video Transformer with shot queries performs the joint estimation.
- Synthetic transition pipeline: To avoid imprecise manual labeling, a fully synthetic transition synthesis pipeline automatically reproduces major transition families with precise boundaries and parameterized variation.
- OmniShotCutBench: A modern wide-domain benchmark enabling holistic and diagnostic evaluation of SBD methods.