Overview
- Field: Computer Vision
- Authors: Zhenghao Xing, Ruiyang Xu, Yuxuan Wang
- Published: 2026-06-19
- arXiv: 2506.14987
Summary
Passive models for long video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with video length.
OmniAgent, the first native omni-modal agent, formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration.
Key Components
1. Agentic Supervised Fine-Tuning: Bootstraps native active perception via best-of-N trajectory synthesis with dual-stage quality control. 2. Agentic Reinforcement Learning: Employs TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level entropy to steer credit assignment toward pivotal discovery turns.
Results
Crucially, OmniAgent exhibits positive test-time scaling—performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate state-of-the-art performance among open-source models. Notably, on LVBench, the 7B agent outperforms the 10x larger Qwen2.5-VL-72B (50.5% vs. 47.3%).
---
*Auto-collected on 2026-06-19*