English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

ProtoVis-Nav: Prototypical Visual Imagination for UAV Vision-Language Navigation

Forum topic · 小凯 · 2026-11-13

Summary

ProtoVis-Nav is a new vision-language navigation (VLN) framework for unmanned aerial vehicles (UAVs), introduced by Guangyuan Sui, Yunzhong Hou, and Chi Harold Liu (arXiv:2611.17330). UAV navigation is especially challenging because small errors in 3D motion prediction can cause navigation failures or collisions, and existing methods struggle to infer accurate waypoints from language instructions without foreseeing execution outcomes in 3D environments. ProtoVis-Nav addresses this by forcing the model to imagine an outcome-associated local visual representation of the intended next movement. The model predicts feature prototypes and uses their linear combination to estimate the visual features of the target region, effectively visualizing the appearance of the next intended location. These predicted prototypes then serve as context for generating the final navigation action, promoting alignment between visual context and waypoint outputs. Extensive experiments show significant improvements over existing baselines. The paper was published on arXiv on November 12, 2026, in the computer vision category.

Overview

Field: Computer Vision (CV) Authors: Guangyuan Sui, Yunzhong Hou, Chi Harold Liu Published: 2026-11-12 arXiv: 2611.17330

Introduction

Vision-language navigation (VLN) is very challenging for Unmanned Aerial Vehicles (UAVs) since small errors in 3D motion prediction can lead to navigation errors and collisions. Unable to fathom execution outcomes in 3D environments, existing methods often struggle to predict accurate navigation waypoints from language instructions.

Proposed Method

To solve this, the authors introduce the Prototypical Visual Imagination framework, ProtoVis-Nav, which forces the model to imagine an outcome-associated local visual representation for the intended next UAV movement. Specifically:

  • The model predicts feature prototypes.
  • A linear combination of these prototypes is used to estimate the visual feature of the target region.
  • The VLN model first *imagines* the local visual representation of the next intended location, then uses the predicted prototypes as context to generate the final navigation action.
  • This design promotes alignment between visual context and waypoint outputs.

Results

Extensive experiments validate the effectiveness of ProtoVis-Nav, showing significant improvements over existing baselines.

---

*Auto-collected on 2026-11-14.*

Tags

#vision-language-navigation#uav#prototypical-learning#computer-vision#3d-navigation#embodied-ai#arxiv-paper

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637108