SR-ReaL: Dual-Path Reasoning for Spatial Vision-Language Models
> Paper: Reinforcing Dual-Path Reasoning in Spatial Vision Language Models > Authors: Yatai Ji, An-Chieh Cheng, Yang Fu, et al. (HKU, NVIDIA, UCSD) > Link: https://arxiv.org/abs/2606.17539 > Core contribution: First spatial VLM to simultaneously support Language-Only Reasoning (LOR) and Detect-Then-Reason (DTR) paths, mutually enhanced within a unified reinforcement learning framework.
The Problem: One Question Type, Two Solutions
Spatial VLMs have progressed quickly but still struggle with complex spatial reasoning — not simple one-step relation identification ("the cat is on the sofa"), but multi-step questions like:
- "How many doors do you pass walking from the living room to the bedroom?"
- "How far is this chair from the table? What obstacles must you go around?"
- "From this viewpoint, is object A to the left or right of object B?"
- DTR data improves LOR (explicit geometry strengthens underlying spatial representations).
- LOR data improves DTR (prevents over-reliance on quantitative computation, restoring "spatial intuition").
- Joint training wins on both paths.
- Removing the detection reward raises 3D localization error from 0.45 to 0.78.
- Cold-start SFT teaches CoT format but generalizes poorly; RL-only scores well but produces logically inconsistent CoT; the combined pipeline achieves the best results (e.g., 72.21 on SPAR single-view vs. 38.97 baseline).
- Mainly validated in single-view, indoor settings; multi-view and outdoor (NuScenes) coverage is limited.
- Some subtasks (viewpoint change) remain below 50% absolute accuracy.
- Training requires 32 A100 GPUs.
- Ji et al., "Reinforcing Dual-Path Reasoning in Spatial Vision Language Models", arXiv:2606.17539, 2026
- Cheng et al., "SpatialRGPT", 2024
- Cheng et al., "SR-3D", 2025
- DeepSeek-AI, "DeepSeek-R1", 2025
- Zhang et al., "SPAR", 2025
The authors observe two phenomena:
1. Some problems are solvable purely by language reasoning, e.g., "A is left of B, B is right of C — where is A relative to C?" Step-by-step textual deduction suffices. 2. Some problems require explicit 3D geometry, e.g., "How many meters apart are these two objects?" Quantitative questions demand detected 3D coordinates first.
All existing frameworks support only one of these strategies. None support both — let alone mutual enhancement.
Core Design: Dual-Path Complementary Reasoning
Path 1: LOR (Language-Only Reasoning)
Pure step-by-step linguistic deduction, suited to relational derivation, viewpoint reasoning, and spatial imagination.
Path 2: DTR (Detect-Then-Reason)
Detect first, then reason. Region tokens are used to extract 3D coordinates (center point or bounding box), enabling quantitative reasoning about depth, distance, and precise localization.
Key insight: the two paths are complementary, not mutually exclusive. A single model learns both and selects via prompt instructions.
Region-to-3D Grounding
Directly predicting 3D coordinates from text is extremely difficult. SR-ReaL bridges this with region tokens:
1. A natural-language target description (e.g., "the red chair") is mapped to a visual region (mask/bounding box) by a region branch, producing region tokens.
2. Region tokens are decoded into a 3D center (x, y, z) or a 3D bounding box (center + size + orientation).
3. Explicit 3D coordinates feed geometric computation for the final answer.
This decouples semantic parsing from 3D perception. Ablations confirm it: removing the region-to-3D interface drops DTR from 60.6 to 59.3 and doubles 3D localization error (0.45 → 0.67).
Two-Stage Training: Cold-Start SFT + RL
Stage 1: Cold-Start SFT
Gives the SR-3D base model basic reasoning and grounding capability (~1M samples total):
| Data | Amount | Purpose | |------|--------|---------| | CoT-LOR | 30k | Language chain-of-thought reasoning | | CoT-DTR | 10k | Detection-then-reasoning with 3D cues | | Complex spatial tasks | 20k | Navigation, interaction, layout reasoning | | 2D Grounding | RefCOCO | Text-to-2D-box prediction | | 3D Grounding | Omni3D / OmniNOCS | Region-to-3D mapping | | Region-prompt QA | SRGPT | Local spatial understanding | | General multimodal QA | LLaVA-1.5 | Prevent overfitting |
Training on CoT data alone causes rapid degradation of general multimodal ability; mixing general data is a prerequisite for stable RL.
Stage 2: Reinforcement Learning
DAPO-style GRPO with three rewards:
1. Format reward:
<think>...</think>
<answer>...</answer> for LOR;
<detect>...</detect>
<think>...</think>
<answer>...</answer> for DTR.
2. Accuracy reward: binary for multiple choice; exponentially smoothed relative error for numeric answers.
3. 3D detection reward (DTR only): discretized reward based on distance between predicted 3D center and ground truth.
An online filtering mechanism (borrowed from DAPO) removes rollout groups with zero relative advantage, then resamples to keep batch size.
Results
SPAR-Bench (20 subtasks)
| Model | SPAR-Bench avg | EmbSpatial | SAT | |-------|---------------|-----------|-----| | SR-3D (baseline) | 33.4 | 72.5 | 63.0 | | Ours-LOR | 60.5 | 79.2 | 68.7 | | Ours-DTR | 61.9 | 81.3 | - |
DTR nearly doubles the baseline (+28.5).
Mutual Reinforcement
| Training | SPAR (LOR) | SPAR (DTR) | EmbSpatial (LOR) | EmbSpatial (DTR) | |----------|-----------|-----------|------------------|------------------| | LOR only | 58.0 | - | 75.9 | - | | DTR only | - | 57.2 | - | 71.4 | | Joint | 58.7 | 60.8 | 77.6 | 78.8 |
Ablations
Out-of-Distribution
On BLINK, RealWorldQA, and CVBench, direct answering (no CoT) outperforms CoT in OOD settings (e.g., 87.4 on BLINK spatial vs. 83.9 baseline), showing the model retains its original spatial perception and can flexibly choose whether to reason step-by-step.
Technical Highlights
1. Unified dual-path design: one model, one checkpoint, prompt-guided routing between LOR and DTR. 2. RL shapes reasoning behavior, not just answers: tighter LOR chains, better attention allocation in DTR, mutual regularization. 3. Data quality > quantity: only ~40k CoT samples, but strictly filtered (answer match + LLM-verified logical consistency). 4. Full pipeline from perception to reasoning: image + question → semantic parsing → region token → 3D coordinates → geometric reasoning → answer.
Limitations and Future Work
Future directions: video input and temporal reasoning, more than two reasoning paths, and integration with world models for spatial planning.
Conclusion
SR-ReaL marks a paradigm shift from "single reasoning strategy" to "multi-path complementary reasoning" — an idea transferable to mathematics (symbolic derivation vs. numerical verification), science, and everyday reasoning. As the paper puts it:
> "Jointly training both reasoning modes fosters mutual reinforcement, with each mode benefiting from the other's supervision."
Not 1+1=2, but an emergent 1+1>2.