TimeProVe: Propose-then-Verify Framework for Efficient Long Video Temporal Reasoning in Daily Activities

Forum topic · 小凯 · 2026-06-22

Summary

Long video question answering (LVQA) requires locating sparse, query-relevant evidence in hours-long untrimmed video, but existing approaches either densely process video with expensive vision-language models (VLMs) or rely on caption-based reasoning that misses localized, action-centric evidence. TimeProVe is a cost-efficient hybrid framework for temporal grounding: a lightweight module first generates action-based answer-evidence hypotheses, and the expensive VLM is invoked only for targeted verification. Its core Action-based Candidate Evidence (ACE) module uses lightweight LLM reasoning to convert temporally localized actions into query-conditioned candidate answers with supporting evidence windows. The authors also introduce OpenTSUBench (OTB), an open-ended benchmark for evaluating temporal grounding in real-world activities of daily living (ADL) scenarios. Experiments show TimeProVe outperforms the strongest baseline on OTB by 7.3% while reducing VLM calls by 75% and inference cost by 93%. Without explicit temporal grounding training, it also achieves competitive results on Charades-STA, and reaches state-of-the-art performance when combined with a grounding VLM. Paper: arXiv 2506.17587.

Paper Overview

Field: cs.CV
Authors: Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan
Published: 2026-06-21
arXiv: 2506.17587

Abstract

Long video question answering (LVQA) requires finding sparse, query-relevant evidence in untrimmed videos lasting several hours. Existing methods either densely process video with large vision-language models (VLMs) at prohibitive computational cost, or rely on sparse caption-based reasoning that often misses temporally localized, action-centric evidence.

TimeProVe is a cost-efficient hybrid framework for temporal grounding in long videos. It first uses lightweight modules to generate action-based answer-evidence hypotheses, then invokes the expensive VLM only for targeted verification when needed.

At the core of the framework is the Action-based Candidate Evidence (ACE) module, which converts temporally localized actions into query-conditioned candidate answers and supporting evidence windows via lightweight LLM reasoning.

The authors also introduce OpenTSUBench (OTB), an open-ended benchmark designed to evaluate temporal grounding in real-world activities of daily living (ADL) scenarios.

Key Results

Outperforms the strongest baseline on OTB by 7.3%
Reduces VLM calls by 75% and inference cost by 93%
Achieves competitive performance on Charades-STA even without explicit temporal grounding training
Reaches state-of-the-art results when combined with a grounding VLM

Plain-Language Explanation

Imagine searching a ten-hour surveillance video for the answer to "who put the keys in the drawer at hour 3, minute 37?" Previous approaches either feed the entire footage to an extremely expensive AI model, or rely on rough human-written captions and miss the crucial actions.

TimeProVe's approach: first send cheap "detectives" (lightweight modules) to quickly scan the video and flag potentially relevant action segments, producing a shortlist of "suspect + evidence window" hypotheses. Only then is the expensive lead detective (the large VLM) brought in to verify the final answers. Like a detective who doesn't stare at surveillance footage all day but focuses on the few minutes that matter most — saving effort without missing clues.