UniD: Unified Video Dense Prediction from Disjoint Data

Forum topic · 小凯 · 2026-07-25

Summary

UniD is a unified video model that jointly predicts eight dense scene properties—depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials—from disjoint, domain-specific datasets. Existing unified systems typically require fully co-annotated data or costly pseudo-labeling; UniD avoids both via a simple distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating any need for annotation overlap or pseudo-labels. The key insight is that strong visual priors from pre-trained diffusion models are sufficient to bridge the domain gap introduced by disjoint training sources, enabling robust generalization to scene-task combinations never seen during training. UniD matches task-specific experts and multi-task baselines across all eight tasks, generalizes strongly to out-of-distribution scenes, and improves temporal consistency and cross-task consistency. Paper: arXiv 2507.19319.

Overview

Field: Computer Vision Authors: Yihong Sun, Seoung Wug Oh, Jiahui Huang arXiv: 2507.19319

Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific datasets. Current unified systems circumvent this by restricting training to fully co-annotated data, or by incurring the large computational cost of pseudo-labeling.

Key Contributions

UniD: a unified video model that jointly predicts eight dense scene properties — depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials — all learned from disjoint, domain-specific datasets.
Expert distillation: a simple yet effective distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo-labeling.
Core insight: the strong visual priors of pre-trained diffusion models are sufficient to bridge the domain gap introduced by disjoint training sources, enabling robust generalization to scene–task combinations never seen during training.

Results

UniD achieves performance comparable to task-specific experts and multi-task baselines across all tasks, shows strong generalization to out-of-distribution scenes, and enhances temporal consistency and cross-task consistency.