Paper Overview
- Field: Computer Vision
- Authors: Wanrong Zheng, Yunhao Ge, Laurent Itti
- Published: 2025-04-30
- arXiv: 2504.20756
- Hierarchical global-local planner operating in three perspectives (global plan, local alignment, trajectory audit).
- Requires no gradient updates or task-specific fine-tuning — it plugs into existing VLN pipelines with minimal overhead.
- Achieves state-of-the-art zero-shot performance on the R2R-CE and RxR-CE datasets.
- Addresses two common failure modes of MLLM-based zero-shot VLN agents: course drift and premature stopping.
- Code: https://github.com/ZoeyZheng0/3-step-Nav
- Paper: https://arxiv.org/abs/2504.20756
Abstract
Breakthrough progress in vision-based navigation through unknown environments has been achieved by using multimodal large language models (MLLMs). These models can plan a sequence of motions by evaluating the current view at each time step against the task and goal given to the agent. However, current zero-shot Vision-and-Language Navigation (VLN) agents powered by MLLMs still tend to drift off course, halt prematurely, and achieve low overall success rates.
The authors propose Three-Step Nav to counteract these failures with a three-view protocol:
1. Look forward — extract global landmarks and sketch a coarse plan. 2. Look now — align the current visual observation with the next sub-goal for fine-grained guidance. 3. Look backward — audit the entire trajectory to correct accumulated drift before stopping.