Paper Overview
- Field: cs.CV
- Authors: Like Liu, Zhengzheng Xu, Haitao He
- Published: 2026-07-21
- arXiv: 2507.15492
- Dual-cognition evaluation: UAV-DualCog includes both image and video tasks to jointly evaluate self-state and environment-state reasoning, while requiring spatial or temporal grounding beyond discrete answer prediction.
- Automated data pipeline: Data is constructed from scene-level semantic point clouds, yielding a scalable benchmark with diverse scenes, hundreds of landmarks, and thousands of QA samples.
- Findings: Extensive evaluations show that current MLLMs remain far from reliable in UAV dual cognition. Self-state reasoning, viewpoint transformation, precise spatial grounding, and temporal interval localization are persistent bottlenecks.
- Validation: Additional testing with thinking/frontier models and a human baseline confirms the benchmark is understandable to humans but challenging for existing models.
- Training resource: A training set, UAV-DualCog-Train, is built from disjoint scenes. A lightweight optimization probe shows it provides useful structured supervision, suggesting value not only as an evaluation benchmark but also as a data resource for advancing MLLM-based UAV agents.
- Paper: arXiv:2507.15492
- Auto-collected on 2026-07-21
Abstract
Multimodal large language models have achieved strong performance across diverse vision-language tasks, yet their capabilities in UAV scenarios remain insufficiently explored. Recent UAV-oriented benchmarks have begun to evaluate MLLMs in aerial scenarios, but they typically focus on scene understanding, event recognition, or navigation completion, rather than jointly assessing the dual-cognition capability required for UAV agents: reasoning about both the UAV's own state and the external environment in multiview spatio-temporal contexts.To address this gap, the authors present UAV-DualCog, a benchmark for aerial multiview spatio-temporal reasoning built on this dual-cognition perspective.