UNIEGO: Proxies as Mediators for Unified Egocentric Video Representation Learning

Forum topic · 小凯 · 2026-06-21

Summary

UNIEGO is a unified egocentric video encoder trained via hierarchical multi-teacher distillation, addressing the limited field of view of wearable cameras. The model learns from 9 teachers spanning egocentric and third-person viewpoints, RGB, depth, and skeleton modalities, plus 4 foundation models. Because direct distillation from heterogeneous teachers produces conflicting gradients due to incompatible architectures and feature geometries, the framework inserts representation-specific proxy models that translate diverse teacher knowledge into a homogeneous egocentric space. A second-stage Selective Proxy Distillation (SPD) adaptively selects proxies that are both correct and confident for each training sample, distilling only reliable supervision. SPD further stabilizes training by initializing UNIEGO as a learned convex combination of proxy parameters. UNIEGO achieves state-of-the-art results on action recognition, video retrieval, and action segmentation across three egocentric/third-person benchmarks, outperforming naive multi-teacher distillation baselines and demonstrating that structured, proxy-mediated knowledge transfer yields richer, more discriminative egocentric representations. Source: arXiv 2506.16620.

Paper Overview

Research Area: Computer Vision Authors: Wenhao Chi, Arkaprava Sinha, Dominick Reilly Published: 2026-06-20 arXiv: 2506.16620

Summary

Egocentric video understanding is fundamentally limited by the narrow viewpoint of wearable cameras: a single viewpoint, single modality, and single model cannot capture the full richness of human actions. This paper argues that truly expressive egocentric representations must integrate complementary knowledge across viewpoints, modalities, and foundation models, while remaining deployable with only egocentric video input.

The authors introduce a hierarchical multi-teacher distillation framework to train UNIEGO, a unified egocentric encoder trained with 9 teachers covering egocentric/third-person viewpoints, RGB, depth, and skeleton modalities, plus 4 foundation models.

Key Contributions

Proxy-mediated distillation: Rather than distilling directly from heterogeneous teachers — whose incompatible architectures and feature geometries generate conflicting gradients — the framework inserts representation-specific proxy models that transform diverse teacher knowledge into a homogeneous egocentric space.
Selective Proxy Distillation (SPD): A second stage that adaptively selects a subset of proxies that are both correct and confident for each training sample, distilling only reliable supervision and suppressing erroneous signals.
Stabilized initialization: SPD further stabilizes training by initializing UNIEGO as a learned convex combination of proxy parameters, placing the unified model in a well-conditioned loss region before distillation begins.

Results

UNIEGO achieves state-of-the-art performance on three egocentric video understanding tasks — action recognition, video retrieval, and action segmentation — across three challenging egocentric/third-person benchmarks, outperforming naive multi-teacher distillation baselines. This demonstrates that structured, proxy-mediated knowledge transfer produces richer, more discriminative egocentric representations.

--- *Auto-collected on 2026-06-21*

UNIEGO: Proxies as Mediators for Unified Egocentric Video Representation Learning

Summary

Paper Overview

Summary

Key Contributions

Results

Tags