Paper Overview
Field: Computer Vision (CV) Authors: Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan arXiv: 2506.16808
Summary
Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches either process videos densely with large vision-language models (VLMs), incurring prohibitive computational cost, or rely on sparse caption-based reasoning, which often misses temporally localized and motion-centric evidence.
The paper introduces TimeProVe, a cost-efficient hybrid framework for temporally grounded reasoning in long videos. TimeProVe first employs lightweight modules to generate action-grounded answer-evidence hypotheses, then invokes an expensive VLM only for targeted verification.
Key Components
- Action-based Candidate Evidence (ACE) module: the core of the framework, which converts temporally localized actions into query-conditioned candidate answers and supporting evidence windows using lightweight LLM reasoning.
- Targeted VLM verification: expensive VLM calls are restricted to verifying only the most promising hypotheses.
- OpenTSUBench (OTB): a new open-ended benchmark for evaluating temporal grounding reasoning in real-world daily activity (ADL) scenarios.
- Outperforms the strongest baseline on OTB by 7.3%.
- Reduces VLM calls by 75% and inference cost by 93%.
- Without explicit temporal grounding training, achieves competitive performance on Charades-STA; when augmented with a grounding VLM, reaches state-of-the-art results.
Results
Original Abstract (excerpt)
> Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches either process videos densely with large vision-language models (VLMs), incurring prohibitive computational cost, or rely on sparse caption-based reasoning, which often misses temporally localized and motion-centric evidence. We introduce TimeProVe, a cost-efficient hybrid framework for temporally grounded reasoning in long videos...
Paper: https://arxiv.org/abs/2506.16808