TimeProVe: Propose-then-Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living

This post introduces TimeProVe, a hybrid framework proposed in a June 2026 arXiv paper by Arkaprava Sinha, Dominick Reilly, and Siddharth Krishnan of the University of Central Florida. It targets long video question answering (LVQA), using the relatable scenario of a security guard who must find a few seconds of relevant footage among dozens of hours of surveillance video.

Key points

• The problem: sparse evidence in long videos. In an 8-hour ADL recording, the relevant event may occupy only ~0.3% of frames. Frame-by-frame VLM analysis ("dense processing") is computationally prohibitive, while sparse caption-based reasoning ("ants approach") often misses temporally localized and motion-centric evidence — captions say "someone entered the kitchen" but not when, how, or what they did.
• Core idea: Propose, then Verify. TimeProVe mirrors Sherlock Holmes-style reasoning (and Kahneman's System 1 / System 2): lightweight modules quickly form answer-evidence hypotheses, and an expensive VLM is invoked only for targeted verification of specific time windows.
• ACE module (Action-based Candidate Evidence):
1. Action detection — lightweight models scan the video and detect action events (e.g., "Person A picked up the vase"). 2. Query conditioning — the question is converted into keywords, relevant actions, and time windows. 3. Candidate generation — outputs candidate answers with supporting evidence windows and confidence levels. 4. VLM verification — the VLM examines only the proposed windows to confirm or refute the hypothesis.
• Benchmark results (OpenTSUBench / OTB): The paper introduces OTB, a new benchmark for temporally localized reasoning in ADL. TimeProVe reportedly leads the strongest baseline by +7.3% accuracy, with 75% fewer VLM calls and 93% lower inference cost.
• Zero-shot temporal grounding: Without explicit temporal localization training, TimeProVe achieves competitive results on Charades-STA, suggesting ACE's detect-condition-locate pipeline is a general temporal grounding mechanism. With an added grounding VLM, it reaches state-of-the-art results.

Potential applications

• Medical monitoring: locating symptom onset in hours of ICU footage.
• Industrial safety: flagging safety-violation segments in factory surveillance.
• Elder care: extracting medication-related clips from a full day of home video.

Takeaway

TimeProVe bridges the gap between sparse (fast but coarse) and dense (accurate but costly) long-video methods by using each for what it does best: sparse screening followed by dense, targeted verification.

References

[1] Sinha, A., Reilly, D., & Krishnan, S. (2026). *TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living*. arXiv preprint. https://arxiv.org/abs/2606.XXXXX

[2] Gao, J., et al. (2018). CTAP: Complementary temporal action proposal. *AAAI*.

[3] Zhang, H., et al. (2020). Span-based localizing network for natural language video localization. *CVPR*.

[4] Mun, J., et al. (2020). Local-Global Context-Aware Transformer for Language-Guided Video Segmentation. *arXiv*.

[5] Kahneman, D. (2011). *Thinking, Fast and Slow*. Farrar, Straus and Giroux.

*Auto-collected on 2026-06-22*