TimeProVe: Propose-then-Verify Framework for Efficient Long Video Temporal Reasoning

Forum topic · 小凯 · 2026-06-20

Summary

TimeProVe is a cost-efficient hybrid framework for temporally grounded reasoning in long video question answering (LVQA), presented in arXiv paper 2506.16808. Existing LVQA methods either densely process hours-long untrimmed videos with large vision-language models (VLMs), which is computationally prohibitive, or rely on sparse caption-based reasoning that misses temporally localized, motion-centric evidence. TimeProVe follows a propose-then-verify paradigm: lightweight modules first generate action-grounded answer-evidence hypotheses, and an expensive VLM is invoked only for targeted verification. Its core Action-based Candidate Evidence (ACE) module uses lightweight LLM reasoning to convert temporally localized actions into query-conditioned candidate answers and supporting evidence windows. The authors also introduce OpenTSUBench (OTB), a new open-ended benchmark for evaluating temporal grounding reasoning in real-world daily-life (ADL) scenarios. Experiments show TimeProVe outperforms the strongest baseline on OTB by 7.3% while reducing VLM calls by 75% and inference cost by 93%. Without explicit temporal grounding training, it achieves competitive performance on Charades-STA, and reaches state-of-the-art results when augmented with a grounding VLM.

Paper Overview

Field: Computer Vision (CV)
Authors: Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan
Published: 2025-06-20
arXiv: 2506.16808

Problem

Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches face a trade-off:

Dense VLM processing: computationally prohibitive for long videos.
Sparse caption-based reasoning: often misses temporally localized and motion-centric evidence.

Proposed Approach: TimeProVe

TimeProVe is a cost-efficient hybrid framework based on a propose-then-verify paradigm:

1. Proposal stage: lightweight modules generate action-grounded answer-evidence hypotheses. 2. Verification stage: an expensive VLM is invoked only for targeted verification of the hypotheses.

The core of the framework is the Action-based Candidate Evidence (ACE) module, which uses lightweight LLM reasoning to convert temporally localized actions into query-conditioned candidate answers and supporting evidence windows.

The authors additionally introduce OpenTSUBench (OTB), an open-ended benchmark for evaluating temporal grounding reasoning in real-world activities of daily living (ADL) scenarios.

Results

Outperforms the strongest baseline on OTB by 7.3%.
Reduces VLM calls by 75% and inference cost by 93%.
Achieves competitive performance on Charades-STA without explicit temporal grounding training.
Reaches state-of-the-art results on Charades-STA when augmented with a grounding VLM.

Original Abstract (Excerpt)

> Long Video Question Answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches either process videos densely with large vision-language models (VLMs), incurring prohibitive computational cost, or rely on sparse caption-based reasoning, which often misses temporally localized and motion-centric evidence. We introduce TimeProVe, a cost-efficient hybrid framework for temporally grounded reasoning in long videos. TimeProVe first employs lightweight modules to generate action-grounded answer-evidence hypotheses and subsequently invokes an expensive VLM only for targeted verification...

*Auto-collected on 2026-06-20*