Paper Overview
Field: Computer Vision Authors: Ye-Chan Kim, Seunghee Choi, SeungJu Cha arXiv: 2509.04290
Summary
Weakly-Supervised Dense Video Captioning (WSDVC) aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level captions per video. Recent approaches synthesize auxiliary transition captions via LLMs to provide additional vision-language alignment, but these synthetic captions lack visual grounding and are rigidly assigned to every inter-event gap at a fixed location and duration.
To address these issues, the authors propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguistic guidance only where warranted. The key components are:
- Frame-level narration: A VLM generates frame-level narratives for the inter-event gaps between events.
- Transition detection: Transitions are detected from semantic variation across these narratives.
- Temporal refinement: For identified transitions, inter-event temporal masks are refined by blending the temporal midpoint with the semantic change point, selecting the width that maximizes vision-language alignment.
- Paper: https://arxiv.org/abs/2509.04290
Results
Experiments on ActivityNet Captions and YouCook2 demonstrate that SBS achieves state-of-the-art performance in both caption generation and event localization.