OmniShotCut: Holistic Relational Shot Boundary Detection with a Shot-Query-Based Dense Video Transformer

Forum topic · 小凯 · 2026-04-29

Summary

OmniShotCut (arXiv:2504.20683) is a computer vision paper proposing a new approach to Shot Boundary Detection (SBD), the task of automatically identifying shot changes and partitioning videos into coherent shots. Existing state-of-the-art methods often produce non-interpretable boundaries, miss subtle but harmful discontinuities, and rely on noisy annotations and outdated benchmarks. OmniShotCut reformulates SBD as structured relational prediction, jointly estimating shot ranges together with intra-shot and inter-shot relations using a shot query-based dense video Transformer. To avoid imprecise manual labeling, the authors introduce a fully synthetic transition synthesis pipeline that automatically reproduces major transition families with precise boundaries and parameterized variations. They also present OmniShotCutBench, a modern wide-domain benchmark enabling holistic and diagnostic evaluation of SBD models. The paper is authored by Boyang Wang, Guangyi Xu, and Zhipeng Tang, released April 29, 2025.

Overview

Field: Computer Vision
Authors: Boyang Wang, Guangyi Xu, Zhipeng Tang
Released: 2025-04-29
arXiv: 2504.20683

Abstract

Shot Boundary Detection (SBD) aims to automatically identify shot changes and divide a video into coherent shots. While SBD was widely studied in the literature, existing state-of-the-art methods often produce non-interpretable boundaries on transitions, miss subtle yet harmful discontinuities, and rely on noisy, low-diversity annotations and outdated benchmarks. To alleviate these limitations, the authors propose OmniShotCut to formulate SBD as structured relational prediction, jointly estimating shot ranges with intra-shot relations and inter-shot relations via a shot query-based dense video Transformer. To avoid imprecise manual labeling, the method adopts a fully synthetic transition synthesis pipeline that automatically reproduces major transition families with precise boundaries and parameterized variations. The paper also introduces OmniShotCutBench, a modern wide-domain benchmark for holistic and diagnostic evaluation.

Key Contributions

1. Relational formulation of SBD — reframes shot boundary detection as structured relational prediction rather than simple per-frame classification. 2. Shot query-based dense video Transformer — jointly estimates shot ranges, intra-shot relations, and inter-shot (cross-shot) relations. 3. Fully synthetic transition pipeline — avoids imprecise manual annotation by automatically generating major transition families with exact ground-truth boundaries. 4. OmniShotCutBench — a modern, wide-domain benchmark enabling both holistic and diagnostic evaluation of SBD models.

---

*Auto-collected on 2026-04-29.*