English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation

Forum topic · 小凯 · 2026-08-13

Summary

DreamFly, proposed by Yan Deng and Fei Xu, is a vision-language-action (VLA) framework for aerial vision-language navigation (VLN) that addresses three core limitations of prior models: short historical memory, short planning horizons, and the inability to know when to stop. First, it introduces a causally aligned historical memory that strictly restricts decisions at time t to observations earlier than t, preventing future-information leakage during training and yielding genuine navigation policies that generalize to unseen environments. Second, it applies a receding-horizon diffusion planner (Plan-K, Execute-One): a diffusion model samples a K-step future action sequence, only the first action is executed, and planning is re-run with fresh visual feedback, handling partial observability and multimodal uncertainty. Third, LiteStop decouples termination from action generation by estimating stop probability directly from an all-masked initial state, reducing circling near goals. On the OpenFly aerial VLN benchmark, DreamFly achieves 32.04% success rate (SR) and 28.22% SPL in seen environments and 29.46% SR and 23.54% SPL in unseen environments, outperforming all compared baselines with the lowest navigation error.

DreamFly: When Drones Learn to Remember and Foresee

> *"Humans are unique not because we can fly, but because we can imagine ourselves flying."* — Antonio Damasio

This post is a richly illustrated Chinese-language walkthrough of the paper "DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation" (Deng & Xu, arXiv:2608.12308). Below is a structured summary of its key points.

Key points

The problem: aerial VLN is a 3D maze

  • Aerial Vision-Language Navigation asks a drone to follow a natural-language instruction (e.g., "find the red-roofed church") using only onboard camera/microphone, in wind, noise, and partial observability.
  • Compared to ground navigation, aerial VLN adds a top-down flattened viewpoint, dynamic disturbances (wind, battery, propeller noise), and linguistic ambiguity ("fly over that forest" — which forest? how high?).
  • Existing VLA models (RT-2, OpenVLA, Dream-VLA) suffer from three obstacles:
  • 1. Short historical memory — early landmarks in the instruction are forgotten. 2. Short planning horizon — models optimize only the next step. 3. No explicit stopping — models circle the goal or stop early.

    Contribution 1: Causally aligned historical memory

  • Principle: at any decision time *t*, the model may only use observations strictly earlier than *t* — no "future leakage."
  • Implemented via a three-layer structure:
  • 1. Observation encoding: each frame is encoded into a visual feature vector fused with the language instruction. 2. Causal temporal alignment: a masking mechanism (analogous to autoregressive attention, applied along time) blocks all observations with timestamp ≥ t. 3. Dynamic memory augmentation: a learned weighting mechanism emphasizes task-relevant history and fades stale information.
  • Effect: unlike conventional memory models whose test success rates drop sharply relative to training (a symptom of "cheating" on future observations), DreamFly learns genuinely causal navigation strategies that transfer to unseen maps.
  • Contribution 2: Receding-horizon diffusion planning (Plan-K, Execute-One)

  • One-shot planners (A*, RRT) fail in partially observable aerial settings: the drone cannot see around corners, wind changes, and target descriptions are ambiguous.
  • DreamFly uses a diffusion model as the planner — apt because both image generation and navigation planning are searches for a feasible path through high-dimensional space from a start state to a goal state, and diffusion naturally models multimodal uncertainty (many valid routes to "the church").
  • Strategy: Plan-K, Execute-One
  • 1. Plan: sample a K-step future action sequence from the diffusion model (denoising carves out plausible trajectories, not a single deterministic path). 2. Execute: run only the first action, observe new imagery. 3. Re-plan in a closed loop with fresh visual feedback.

    Contribution 3: LiteStop — explicit, decoupled termination

  • Stopping is treated as an explicit, independent decision, not a byproduct of action generation.
  • Stop probability is estimated directly from the action logits at the diffusion model's all-masked initial state (i.e., before any action decisions), decoupling "should I stop?" from "where should I fly?"
  • Result: less hovering near the goal, fewer premature stops and missed targets, better overall efficiency.
  • Results on OpenFly

    | Method | Test-Seen SR | Test-Unseen SR | Test-Seen SPL | Test-Unseen SPL | |---|---|---|---|---| | Prior best | ~25% | ~22% | ~20% | ~17% | | DreamFly | 32.04% | 29.46% | 28.22% | 23.54% |

  • Roughly +28% success rate in seen environments and +34% in unseen environments over prior best methods, with the lowest navigation error.
  • The larger gains on unseen maps indicate strong generalization.
  • Why it matters

    The post closes by framing DreamFly's three principles as broadly applicable to autonomous systems: 1. Causal constraints are a foundation for trustworthy AI in any temporal decision system. 2. Receding-horizon planning is a general strategy for uncertainty in partially observable, dynamic environments (robot manipulation, autonomous driving). 3. Explicit termination — knowing when to stop — is a key capability for reliable, interpretable autonomous agents.

    References

  • Deng, Y., & Xu, F. (2026). *DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation*. arXiv:2608.12308.
  • Driess et al. (2023). *PaLM-E: An Embodied Multimodal Language Model*.
  • Brohan et al. (2023). *RT-2: Vision-Language-Action Models*.
  • Kim et al. (2025). *OpenVLA: An Open Source Vision-Language-Action Model*.
  • Song et al. (2025). *Dream-VLA: Dreaming Visions for VLA*.
  • Ho et al. (2020). *Denoising Diffusion Probabilistic Models*.
  • Anderson et al. (2018). *Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments*.
  • Chen et al. (2024). *OpenFly: A Benchmark for Aerial Vision-Language Navigation*.

Tags

#aerial-vln#diffusion-models#vision-language-action#causal-memory#robot-navigation#reinforcement-learning#drone#openfly-benchmark

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633440