Summary
nuReasoning is a large-scale, reasoning-centric dataset and benchmark for autonomous driving (AD), addressing the lack of reasoning supervision in existing AD datasets that focus mainly on perception, prediction, or planning. The dataset contains 20,000 real-world 20-second clips collected across multiple cities, with synchronized multi-camera images, LiDAR data, HD maps, object annotations, and human-verified reasoning annotations covering spatial reasoning, decision reasoning, and counterfactual reasoning. These annotations target long-tail driving scenarios where vehicles must apply common-sense knowledge, understand spatial relations, infer agent interactions, and make safe decisions. Experiments show that fine-tuning vision-language models (VLMs) on nuReasoning significantly improves driving-specific question-answering performance. Moreover, incorporating reasoning supervision into vision-language-action (VLA) training improves planning performance even when textual reasoning outputs are disabled at inference time. The paper is available on arXiv (2605.31572).
Paper Overview
- Field: Computer Vision (CV)
- Authors: Zhiyu Huang, Johnson Liu, Rui Song, Zewei Zhou, et al.
- Published: 2026-05-29
- arXiv: 2605.31572
- PDF: 2605.31572.pdf
Summary
Reasoning is critical for autonomous driving (AD) in long-tail scenarios, where a vehicle must apply common-sense knowledge, understand spatial relationships, infer agent interactions, and make safe decisions. However, existing AD datasets and benchmarks focus mainly on perception, prediction, or planning, offering limited reasoning supervision for real-world long-tail driving scenarios.
This paper introduces nuReasoning, a large-scale, real-world reasoning-centric AD dataset and benchmark. It contains:
- 20,000 real-world 20-second clips collected across multiple cities
- Synchronized multi-camera images and LiDAR data
- HD maps and object annotations
- Human-verified reasoning annotations across three categories:
- Spatial reasoning
- Decision reasoning
- Counterfactual reasoning
Experimental Findings
- Fine-tuning VLMs on nuReasoning significantly improves driving-specific question-answering performance.
- Incorporating reasoning supervision into VLA training improves planning performance, even when textual reasoning outputs are disabled at inference time.
---
*Auto-collected on 2026-06-02.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177980740