Paper Overview
Field: Computer Vision (CV) Authors: Yu Chen, Caorui Li, Ziyu Xiong Published: 2026-07-22 arXiv: 2507.17085
Abstract (Translation)
Long audio-video reasoning is difficult for omnimodal LLMs because the decisive evidence is often sparse, cross-modal, and too expensive to preserve with uniformly high-fidelity inputs.
OmniReasoner is a tool-use post-training framework for thinking with long audio-video: omnimodal LLMs learn, via supervised fine-tuning and reinforcement learning, to decide whether and where to call a zoom-in tool before answering. OmniReasoner first builds a low-cost global preview of the full stream, then — when needed — calls the zoom-in tool with a requested temporal interval for higher-fidelity visual and audio inspection before answering.
Because the model observes different sampling granularities before and after the call (a sparse global preview vs. a denser local clip), the authors introduce TimeAnchor, which keeps the tool's temporal arguments valid and round-trip consistent across granularities rather than binding them to frame indices at a specific sampling rate.
To make this tool-use behavior trainable without costly manual interval annotation, a temporally-augmented data engine synthesizes trajectories for tool-use post-training via video editing and synthesis.
Experiments across omnimodal and video benchmarks show that OmniReasoner improves answer accuracy and temporal grounding while concentrating high-fidelity computation on informative regions. Code has been open-sourced.
--- *Auto-collected on 2026-07-23*