TimeProVe: Propose-then-Verify Framework for Efficient Long Video Temporal Reasoning in Daily Activities

Forum topic · 小凯 · 2026-06-21

Summary

TimeProVe is a cost-efficient hybrid framework for long video question answering (LVQA) that targets sparse, query-relevant evidence in hours-long untrimmed videos. Instead of processing videos densely with expensive vision-language models (VLMs) or relying on sparse captioning that often misses temporal grounding, TimeProVe uses a lightweight module to generate action-based candidate answers and evidence hypotheses, then invokes a VLM only for targeted verification. Its core Action-based Candidate Evidence (ACE) module converts temporally grounded actions into query-conditioned candidate answers and supporting evidence windows via lightweight LLM reasoning. The authors also introduce OpenTSUBench (OTB), an open-ended benchmark for evaluating temporal grounding reasoning in real-world activities of daily living (ADL) scenarios. Experiments show TimeProVe outperforms the strongest baseline on OTB by 7.3% while reducing VLM calls by 75% and inference cost by 93%. Without explicit temporal grounding training, it achieves competitive results on Charades-STA and reaches state-of-the-art when combined with a grounding VLM. Paper: arXiv 2506.16650.

Paper Overview

Field: Computer Vision (CV) Authors: Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan Published: 2026-06-20 arXiv: 2506.16650

Background

Long video question answering (LVQA) requires identifying sparse, query-relevant evidence within hours-long untrimmed videos. Existing approaches fall into two camps:

Dense processing with large VLMs: computationally prohibitive at scale.
Sparse caption-based reasoning: often misses temporal grounding and motion-centric evidence.

Proposed Method

TimeProVe is a cost-efficient hybrid framework for long video temporal grounding reasoning that follows a propose-then-verify paradigm:

1. A lightweight module generates action-based candidate answers and evidence hypotheses. 2. Expensive VLM calls are reserved only for targeted verification of these hypotheses.

The core of the framework is the Action-based Candidate Evidence (ACE) module, which uses lightweight LLM reasoning to convert temporally grounded actions into query-conditioned candidate answers and supporting evidence windows.

New Benchmark

The authors also introduce OpenTSUBench (OTB), an open-ended benchmark for evaluating temporal grounding reasoning in real-world activities of daily living (ADL) scenarios.

Results

Outperforms the strongest baseline on OTB by 7.3%.
Reduces VLM calls by 75% and inference cost by 93%.
Without explicit temporal grounding training, achieves competitive performance on Charades-STA; combining with a grounding VLM reaches state-of-the-art.

---

*Auto-collected on 2026-06-21.*