TimeProVe: Propose-then-Verify Framework for Efficient Long Video Temporal Reasoning in Daily Living Activities

Forum topic · 小凯 · 2026-06-22

Summary

Long Video Question Answering (LVQA) requires locating sparse, query-relevant evidence in hours-long untrimmed video. Existing approaches either densely process video with large vision-language models (VLMs) at prohibitive computational cost, or rely on sparse caption-based reasoning that often misses temporally localized, action-centric evidence. TimeProVe is a cost-efficient hybrid framework for temporal grounding reasoning: a lightweight module first generates answer-evidence hypotheses from action candidates, then an expensive VLM is invoked only for targeted verification. The core Action-based Candidate Evidence (ACE) module uses lightweight LLM reasoning to convert temporally localized actions into query-conditioned candidate answers and supporting evidence windows. The authors also introduce OpenTSUBench (OTB), an open-ended benchmark for evaluating temporal grounding in real-world activities of daily living (ADL) scenarios. Experiments show TimeProVe outperforms the strongest baseline on OTB by 7.3% while reducing VLM calls by 75% and inference cost by 93%. Without explicit temporal grounding training, it also achieves competitive results on Charades-STA, and reaches state-of-the-art when combined with a grounding VLM. Paper: arXiv 2506.17587.

Paper Overview

Field: cs.CV Authors: Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan Published: 2026-06-21 arXiv: 2506.17587

Abstract

Long Video Question Answering (LVQA) requires finding sparse, query-relevant evidence within hours-long untrimmed videos. Existing methods either densely process the video with a large vision-language model (VLM) at prohibitive computational cost, or rely on sparse caption-based reasoning that often misses temporally localized, action-centric evidence.

The authors propose TimeProVe, a cost-efficient hybrid framework for temporal grounding reasoning in long videos. TimeProVe first uses a lightweight module to generate action-based answer-evidence hypotheses, then invokes the expensive VLM only for targeted verification when needed.

At the core of the framework is the Action-based Candidate Evidence (ACE) module, which uses lightweight LLM reasoning to convert temporally localized actions into query-conditioned candidate answers and supporting evidence windows.

The authors also introduce OpenTSUBench (OTB), an open-ended benchmark designed to evaluate temporal grounding reasoning in real-world activities of daily living (ADL) scenarios.

Experiments show that TimeProVe outperforms the strongest baseline on OTB by 7.3%, while reducing VLM calls by 75% and inference cost by 93%. Additionally, even without explicit temporal grounding training, TimeProVe achieves competitive performance on Charades-STA, and reaches state-of-the-art results when combined with a grounding VLM.

Plain-Language Explanation

Imagine searching a ten-hour surveillance recording to answer: "Who put the keys in the drawer at hour 3, minute 37?"

Previous approaches either feed the entire footage to an extremely expensive AI model in one pass (prohibitively costly), or rely only on rough human-written captions, missing the key actions entirely.

TimeProVe's insight: first dispatch "cheap detectives" (lightweight modules) to quickly scan the video and surface all potentially relevant action segments, generating a shortlist of "suspect + evidence window" hypotheses. Only then is this highly condensed list handed to the "expensive detective" (the large VLM) for final verification.

The result? More accurate answers at 93% lower cost, plus a new benchmark (OTB) specifically for finding concrete action evidence in everyday-life scenarios. Like a detective who doesn't watch surveillance footage continuously, but only stakes out the few most probable minutes — saving effort without missing clues.

--- *Auto-collected on 2026-06-21*