[论文] UNIEGO: Proxies as Mediators for Unified Egocentric Video Representati...

论文概要

研究领域: CV 作者: Wenhao Chi, Arkaprava Sinha, Dominick Reilly 发布时间: 2025-06-20 arXiv: 2506.16806

中文摘要

第一人称视频理解固受穿戴相机狭窄视角所限：单一视点、模态或模型难以捕捉人类行动之全貌。本文论，一真正富有表现力之第一人称表征，必须统摄视点、模态与基础模型表征间之互补知识，同时仍能仅自第一人称视频部署。

为此，作者引入一层次化多教师蒸馏框架，产出UNIEGO——一统一第一人称编码器，以九位教师训练，涵盖自我-外在视点、RGB、深度与骨骼模态，以及四种基础模型。非直接自异质教师蒸馏（其不兼容架构与特征几何易致梯度冲突），框架乃置入表征特定之代理模型层，将多样教师知识转译为同质第一人称空间。第二蒸馏阶段，即选择性代理蒸馏（SPD），对每训练样本自适应选取既正确且自信之代理子集，独自可靠监督蒸馏，抑制错误信号。SPD复以将UNIEGO初始化为代理参数之习得凸组合而稳定，在蒸馏始前置统一模型于损失景观之良条件区域。

UNIEGO在三个第一人称视频理解任务（动作识别、视频检索与动作分割）上，于三个具挑战性之自我-外在基准达成最先进性能，优于朴素多教师蒸馏基线，证实结构化、代理中介之知识迁移能产出更丰富且更具判别力之第一人称表征。

原文摘要

Egocentric video understanding is inherently limited by the narrow perspective of wearable cameras: a single viewpoint, a single modality, a single model cannot capture the full richness of human action. We argue that a truly expressive egocentric representation must subsume complementary knowledge across viewpoints, modalities, and foundation model representations, yet remain deployable from egocentric video alone. To this end, we introduce a hierarchical multi-teacher distillation framework that produces UNIEGO, a unified egocentric encoder trained with nine teachers spanning ego-exo viewpoints, RGB, depth, and skeleton modalities, and four foundation models. Rather than distilling directly from heterogeneous teachers whose incompatible architectures and feature geometries induce conflic...

--- *自动采集于 2026-06-20*

#论文 #arXiv #CV #小凯