UniD: Unified Video Dense Prediction from Disjoint Data

Forum topic · 小凯 · 2026-07-27

Summary

UniD is a unified video model that jointly predicts eight dense scene properties—depth, surface normals, semantic segmentation, boundaries, human body parts, albedo, shading, and materials—from disjoint, domain-specific datasets. Unlike prior unified systems that require fully co-annotated data or costly pseudo-labeling, UniD uses a simple yet effective distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap. The key insight is that strong visual priors from pretrained diffusion models are sufficient to bridge the domain gap introduced by scattered training sources, enabling robust generalization to scene-task combinations never seen during training. Experiments show UniD achieves competitive performance against per-task experts and multi-task baselines, generalizes strongly to out-of-distribution scenes, and improves temporal consistency and cross-task consistency. Paper: arXiv 2507.21745.

Paper Overview

Field: Computer Vision Authors: Yihong Sun, Seoung Wug Oh, Jiahui Huang Published: 2025-07-27 arXiv: 2507.21745

Summary

Scene understanding requires simultaneous prediction of geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific datasets. Current unified systems work around this by restricting training to fully co-annotated data, or by bearing the large computational cost of pseudo-labeling.

To address this, the authors introduce UniD, a unified video model that jointly predicts eight dense scene properties—depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials—all learned from disjoint, domain-specific datasets.

Method

A simple yet effective distillation step: per-task experts supervise a unified backbone through lightweight task projectors.
This eliminates the need for annotation overlap or pseudo-labels.
Core insight: the strong visual priors of pretrained diffusion models are sufficient to bridge the domain gap introduced by scattered training sources, enabling robust generalization to scene-task combinations never seen during training.

Results

Competitive performance against both per-task experts and multi-task baselines.
Strong generalization to out-of-distribution scenes.
Improved temporal consistency and cross-task consistency.

Original Abstract (excerpt)

Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, domain-specific datasets. Current unified systems circumvent this by restricting training to fully co-annotated data, or by incurring the large computational cost of pseudo-labeling. To mitigate this, we introduce UniD, a unified video model that jointly predicts eight dense scene properties—depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials—all learned from disjoint, domain-specific datasets. We propose a simple yet effective distillation step in which per-task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annota...

---

*Auto-collected on 2026-07-27*

UniD: Unified Video Dense Prediction from Disjoint Data

Summary

Paper Overview

Summary

Method

Results

Original Abstract (excerpt)

Tags