Paper Overview
Field: Computer Vision Authors: Wenhao Chi, Arkaprava Sinha, Dominick Reilly Published: 2025-06-20 arXiv: 2506.16806
Summary
Egocentric video understanding is inherently limited by the narrow perspective of wearable cameras: a single viewpoint, a single modality, or a single model cannot capture the full richness of human action. The authors argue that a truly expressive egocentric representation must subsume complementary knowledge across viewpoints, modalities, and foundation model representations, while remaining deployable from egocentric video alone.
To this end, they introduce a hierarchical multi-teacher distillation framework that produces UNIEGO, a unified egocentric encoder trained with nine teachers spanning ego-exo viewpoints, RGB, depth, and skeleton modalities, and four foundation models.
Rather than distilling directly from heterogeneous teachers—whose incompatible architectures and feature geometries induce conflicting gradients—the framework inserts representation-specific proxy models that translate diverse teacher knowledge into a homogeneous egocentric space.
In a second distillation stage, Selective Proxy Distillation (SPD) adaptively selects a subset of proxies that are both correct and confident for each training sample, so that only reliable supervision drives the distillation and erroneous signals are suppressed. SPD is further stabilized by initializing UNIEGO as a learned convex combination of the proxy parameters, placing the unified model in a well-conditioned region of the loss landscape before distillation begins.
Results
UNIEGO achieves state-of-the-art performance on three egocentric video understanding tasks—action recognition, video retrieval, and action segmentation—across three challenging ego-exo benchmarks, outperforming naive multi-teacher distillation baselines. This confirms that structured, proxy-mediated knowledge transfer yields richer and more discriminative egocentric representations.
*Auto-collected on 2026-06-20.*