UniD: Unified Video Dense Prediction from Disjoint Data (arXiv 2507.21745)

Forum topic · 小凯 · 2026-07-27

Summary

UniD is a unified video model that jointly predicts eight dense scene properties — depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials — while learning from disjoint, domain-specific datasets. Unlike prior unified systems that require fully co-annotated data or expensive pseudo-labeling, UniD uses a simple distillation approach in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo-labels. The key insight is that strong visual priors from pre-trained diffusion models are sufficient to bridge the domain gaps introduced by fragmented training sources, enabling robust generalization to scene-task combinations never seen during training. The authors report competitive performance against individual task experts and multi-task baselines, strong out-of-distribution generalization, and improved temporal and cross-task consistency. The paper (arXiv 2507.21745, July 2025) is authored by Yihong Sun, Seoung Wug Oh, and Jiahui Huang.

Paper Overview

Field: Computer Vision Authors: Yihong Sun, Seoung Wug Oh, Jiahui Huang Published: 2025-07-27 arXiv: 2507.21745

Abstract

Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific datasets. Current unified systems circumvent this by restricting training to fully co-annotated data, or by incurring the large computational cost of pseudo-labeling.

To mitigate this, the authors introduce UniD, a unified video model that jointly predicts eight dense scene properties — depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials — all learned from disjoint, domain-specific datasets.

Method

UniD employs a simple yet effective distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo-labels. The core insight is that the strong visual priors of pre-trained diffusion models are sufficient to bridge the domain gaps introduced by fragmented training sources, enabling robust generalization to scene-task combinations never seen during training.

Results

Competitive performance compared to individual task experts and multi-task baselines
Strong generalization to out-of-distribution scenarios
Enhanced temporal consistency and cross-task consistency

---

*Auto-collected on 2026-07-27*