DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation

> *"Humans are unique not because we can fly, but because we can imagine ourselves flying."* — Antonio Damasio

This post is a detailed Chinese-language commentary on the paper DreamFly by Yan Deng and Fei Xu (August 2026, arXiv:2608.12308), which targets Aerial Vision-Language Navigation (VLN): guiding a drone by camera, microphone, and natural-language instructions such as "find the church with the red roof."

Key points

Why aerial VLN is much harder than ground navigation

• Viewpoint: drones see a flattened top-down view, making left/right judgments difficult
• Dynamic environment: wind, air currents, battery drain, and propeller noise constantly disturb perception
• Partial observability: only a small frontal area is visible; nothing behind or around corners
• Ambiguous language: e.g., "fly over that forest" — which forest, how high, and what if it is ten kilometers long?

Vision-Language-Action (VLA) models such as RT-2, OpenVLA, and Dream-VLA map perception directly to actions, but face three obstacles in the air:

1. Short historical memory — early instruction clues are forgotten after a few frames 2. Short planning horizon — models focus on the next step without anticipating future paths 3. No reliable stopping — models orbit the target or stop prematurely

DreamFly's three modules

Built on Dream-VLA, DreamFly adds:

1. Causally Aligned Historical Memory — At any decision time t, the model uses only observations strictly earlier than t. Implementation layers:

• Observation encoding: each image becomes a compact visual feature fused with the language instruction
• Causal temporal alignment: masking ensures information flows only from past to present (analogous to autoregressive attention, applied over time)
• Dynamic memory augmentation: a learned weighting mechanism up-weights task-relevant history and fades outdated observations

This prevents "future-information leakage," so models learn genuine navigation policy rather than implicit prediction of future observations — yielding more stable performance in unseen environments.

2. Receding-Horizon Diffusion Planning (Plan-K, Execute-One)

• Plan-K: the diffusion model generates K-step future action sequences by sampling from a distribution of plausible trajectories, naturally handling multi-modal uncertainty
• Execute-One: only the first action is executed; the drone then re-observes and re-plans
• Closed-loop visual feedback keeps plans grounded in the latest information

3. LiteStop — explicit termination

• Stop probability is estimated directly from action logits at an all-masked initial state of the diffusion model
• Termination is decoupled from action generation, making stopping a lightweight, explicit decision rather than a byproduct
• Result: less circling near targets, fewer premature or missed stops

Results on the OpenFly benchmark

| Method | Test-Seen SR | Test-Unseen SR | Test-Seen SPL | Test-Unseen SPL | |--------|-------------|----------------|---------------|-----------------| | Prior best | ~25% | ~22% | ~20% | ~17% | | DreamFly | 32.04% | 29.46% | 28.22% | 23.54% |

• Success rate improves ~28% (seen) and ~34% (unseen) over prior methods
• Lowest navigation error among all compared methods
• Larger gains on unseen maps demonstrate stronger generalization

Broader implications

The author highlights three general design principles:

1. Causal constraints as a foundation of trustworthy AI — decisions must rely only on genuinely available information 2. Receding-horizon planning as a universal strategy under partial observability, extensible to manipulation, autonomous driving, and human decision-making 3. Explicit termination as a key capability of autonomous systems, improving reliability and interpretability

The post closes with an analogy: like an albatross that remembers wind patterns, anticipates conditions ahead, and knows when to dive for fish, DreamFly translates evolved instincts into mathematical form.

References

• Deng, Y., & Xu, F. (2026). *DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation*. arXiv:2608.12308.
• Driess et al. (2023). *PaLM-E: An Embodied Multimodal Language Model*.
• Brohan et al. (2023). *RT-2: Vision-Language-Action Models*.
• Kim et al. (2025). *OpenVLA: An Open Source Vision-Language-Action Model*.
• Song et al. (2025). *Dream-VLA: Dreaming Visions for VLA*.
• Ho et al. (2020). *Denoising Diffusion Probabilistic Models*.
• Anderson et al. (2018). *Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments*.
• Chen et al. (2024). *OpenFly: A Benchmark for Aerial Vision-Language Navigation*.