DreamFly: Causal Memory and Diffusion Planning for Aerial Vision-Language Navigation

Key points

• Problem framing. Aerial VLN is harder than ground navigation because drones see a flattened top-down view, operate in dynamic winds, observe only a partial forward field, and must disambiguate language instructions such as "fly past the woods." Existing Vision-Language-Action (VLA) models (RT-2, OpenVLA, Dream-VLA) suffer from short memory, short planning horizon, and weak stopping decisions.
• Causally Aligned Historical Memory. Traditional memory modules can leak information from future time steps into current decisions, producing inflated training scores and weak generalization. DreamFly enforces strict temporal causality: at decision time t the model can only attend to observations strictly earlier than t. The memory module combines per-step observation encoding, causal temporal masking, and a dynamic weighting that emphasizes task-relevant history.
• Receding-Horizon Diffusion Planning. Instead of one-shot path planning, DreamFly uses a diffusion model in action space. At each step it plans K future actions (Plan-K) but executes only the first (Execute-One), then re-plans with new observations. The diffusion sampling captures the multi-modal distribution of viable trajectories rather than committing to a single greedy path.
• LiteStop explicit termination. Stopping is decoupled from action generation. A stop probability is estimated directly from the all-mask initial state using the action logits, producing a lightweight, independent termination module that reduces hovering near the goal and premature stops.
• Architecture at a glance. Language instruction + current image → causal memory module → diffusion planner → LiteStop → next action (move forward / turn / ascend / descend / stop).
• Benchmark results (OpenFly).

| Method | Test-Seen SR | Test-Unseen SR | Test-Seen SPL | Test-Unseen SPL | |---|---|---|---|---| | Prior best | ~25% | ~22% | ~20% | ~17% | | DreamFly | 32.04% | 29.46% | 28.22% | 23.54% |

Gains are roughly 28% relative on seen environments and 34% relative on unseen environments, with the lowest navigation error, indicating stronger generalization rather than memorized trajectories.

• Broader implications. The authors distill three general design principles: enforce causal constraints on temporally conditioned decisions, use receding-horizon re-planning under partial observability, and decouple termination from action selection in autonomous systems.

References

• Deng, Y., & Xu, F. (2026). *DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation*. arXiv:2608.12308.
• Driess et al. (2023). *PaLM-E: An Embodied Multimodal Language Model*.
• Brohan et al. (2023). *RT-2: Vision-Language-Action Models*.
• Kim et al. (2025). *OpenVLA: An Open Source Vision-Language-Action Model*.
• Song et al. (2025). *Dream-VLA: Dreaming Visions for VLA*.
• Ho et al. (2020). *Denoising Diffusion Probabilistic Models*.
• Anderson et al. (2018). *Vision-and-Language Navigation*.
• Chen et al. (2024). *OpenFly: A Benchmark for Aerial Vision-Language Navigation*.