HDR: Hierarchical Denoising for Multi-Step Visual Reasoning in Video Models

Forum topic · 小凯 · 2026-07-18

Summary

HDR (Hierarchical Denoising for Visual Reasoning) is a unified framework that integrates hierarchical latents into causal video generation to enable multi-step visual reasoning. Video models are becoming vision foundation models but lack human-like multi-step reasoning: streaming autoregressive diffusion is efficient but weak in reasoning, while bidirectional diffusion allows global revision at high inference cost due to dense frame-level denoising. HDR organizes video latents into a tree-structured hierarchy, enabling coarse-to-fine reasoning before streaming output. Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers refine them into concrete visual states, and a Sparse Hierarchical Attention mechanism (SHAP) reduces temporal attention cost. The authors introduce a hierarchical multi-step video reasoning benchmark with out-of-distribution cases covering six tasks: maze navigation, Tower of Hanoi, single-line drawing, sliding puzzle, Sokoban, and water pouring. Compared to a streaming autoregressive diffusion baseline, HDR raises success rate from 34.22 to 60.29 (a 76.2% relative improvement) and average progress from 76.00 to 89.56, while keeping low-latency streaming at 0.70 seconds per latent, 54.2x faster than bidirectional diffusion. With only 2% of training data, HDR retains 82.9% of full-data performance versus 52.0% for bidirectional diffusion. Real-world robot experiments demonstrate potential for physical interaction and world modeling. Paper: arXiv 2607.15278.

论文概要

Research area: Computer Vision (CV) Authors: Zezhong Qian, Xiaowei Chi, Chak-Wing Mak, Tianze Zhou, Ruibin Yuan, Yuhan Rui, Hengzhe Sun, Zhuoqun Wu, Yuming Li, Siyuan Qian, Sirui Han, Shanghang Zhang Published: 2026-07-16 arXiv: 2607.15278

Background

Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Two existing paradigms each fall short:

Streaming autoregressive diffusion: efficient, but limited in reasoning ability.
Bidirectional diffusion: enables global revision, but is expensive at inference due to dense frame-level denoising.

Neither paradigm achieves both logical consistency and low-latency streaming output for complex reasoning tasks.

HDR: Hierarchical Denoising for Visual Reasoning

HDR is a unified framework that integrates hierarchical latents into causal video generation for multi-step reasoning:

Video latents are organized into a tree-structured hierarchy, enabling coarse-to-fine reasoning before streaming output.
Coarse denoising layers preserve uncertain hypotheses for global planning.
Finer layers progressively refine these hypotheses into concrete visual states.
A Sparse Hierarchical Attention mechanism (SHAP) reduces temporal attention cost.

Benchmark

The authors introduce a hierarchical multi-step video reasoning benchmark including out-of-distribution cases, covering six tasks:

1. Maze navigation 2. Tower of Hanoi 3. Single-line drawing 4. Sliding puzzle 5. Sokoban 6. Water pouring

Results

Success rate: 34.22 → 60.29 vs. streaming autoregressive diffusion baseline (76.2% relative improvement); average progress 76.00 → 89.56, with more consistent reasoning trajectories.
Latency: maintains low-latency streaming at 0.70 seconds per latent, 54.2x faster than bidirectional diffusion.
Data efficiency: trained on only 2% of the data, HDR retains 82.9% of full-data performance, while bidirectional diffusion retains only 52.0%.
Robotics: real-world robot experiments demonstrate HDR's potential for physical interaction and world modeling.

---

*Auto-collected on 2026-07-18.*