HDR: Hierarchical Denoising for Multi-Step Visual Reasoning in Video Models

Forum topic · 小凯 · 2026-07-18

Summary

HDR (Hierarchical Denoising for Visual Reasoning) is a unified framework that integrates hierarchical latents into causal video generation to enable multi-step reasoning, addressing the trade-off between streaming autoregressive diffusion (efficient but weak reasoning) and bidirectional diffusion (globally revisable but computationally expensive). HDR organizes video latents into a tree-structured hierarchy for coarse-to-fine reasoning before streaming output: coarse denoising layers keep uncertain hypotheses for global planning while finer layers refine them into concrete visual states, and a Sparse Hierarchical Attention mechanism (SHAP) reduces temporal attention cost. On a new hierarchical multi-step video reasoning benchmark covering six tasks (maze navigation, Tower of Hanoi, single-line drawing, sliding puzzle, Sokoban, and water pouring), HDR raises success rate from 34.22 to 60.29 (+76.2% relative) and average progress from 76.00 to 89.56 over streaming autoregressive baselines, while maintaining low-latency streaming at 0.70 seconds per latent and running 54.2x faster than bidirectional diffusion. With only 2% of training data, HDR retains 82.9% of full-data performance versus 52.0% for bidirectional diffusion. Real-world robot experiments demonstrate its potential for physical interaction and world modeling. Paper: arXiv 2607.15278.

Overview

Field: Computer Vision (CV) Authors: Zezhong Qian, Xiaowei Chi, Chak-Wing Mak, Tianze Zhou, Ruibin Yuan, Yuhan Rui, Hengzhe Sun, Zhuoqun Wu, Yuming Li, Siyuan Qian, Sirui Han, Shanghang Zhang Published: 2026-07-16 arXiv: 2607.15278

Abstract

Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Streaming autoregressive diffusion models are efficient but limited in reasoning, while bidirectional diffusion enables global revision with high inference costs due to dense frame-level denoising. Both paradigms struggle to achieve logical consistency and low-latency streaming for complex reasoning tasks.

The authors propose HDR (Hierarchical Denoising for Visual Reasoning), a unified framework that integrates hierarchical latents into causal video generation for multi-step reasoning.

Key Ideas

Tree-structured latent hierarchy: HDR organizes video latents hierarchically, enabling coarse-to-fine reasoning before streaming output.
Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers progressively refine them into concrete visual states.
Sparse Hierarchical Attention (SHAP) reduces the cost of temporal attention.

Benchmark and Results

A hierarchical multi-step video reasoning benchmark with out-of-distribution cases was introduced, covering six tasks: maze navigation, Tower of Hanoi, single-line drawing, sliding puzzle, Sokoban, and water pouring.

Success rate: 34.22 → 60.29 vs. streaming autoregressive diffusion baseline (+76.2% relative)
Average progress: 76.00 → 89.56, with more logically consistent reasoning trajectories
Streaming latency: 0.70 seconds per latent
Inference speed: 54.2x faster than bidirectional diffusion
Data efficiency: with only 2% of training data, HDR retains 82.9% of full-data performance (bidirectional diffusion: 52.0%)

Real-world robot experiments further demonstrate HDR's potential for physical interaction and world modeling.