[论文] EgoExplorer: A Vision-Motion-Touch Observatory for Human Loco-Manipula...

论文概要

研究领域: CV
作者: Yunsong Zhou, Xuekun Jiang, Haiqi Shen, Hangxu Liu, Chaojian Shi, Yujie Liu, Jingyuan Li, Haoran Yang, Jiafei Cao, Dongzhan Zhou, Dong Wang, Yao Mu, Chunhua Shen, Weinan Zhang
发布时间: 2026-10-13
arXiv: 2610.18872

中文摘要

从人类演示学习人形机器人移动-操作技能，关键在于观察 locomotion、姿态和物体交互如何随任务推进协同演化。要在多样环境中规模化采集，轻量便携的捕获系统更为有利。然而现有数据集仅提供这种协同的部分视角。为此我们推出 EgoExplorer——一个基于可穿戴系统的 100 小时人类移动-操作「观测台「，侧重专业活动场景，连续记录将自我中心视频与重建的全身运动、精细手指关节运动和触觉观测对齐，并附带步骤级描述。为了在稀疏体感条件下实现这一覆盖度，我们开发了 Calibrate-Complete-Constrain 管线：可复用的标定在运动推理前固定参与者几何和传感器佩戴方式，将安装偏移与关节运动解耦；时序先验补全未观测的关节运动；分层约束促进测量一致性、时序连续性和几何一致性——这些性质仅靠运动合理性无法保证。相对于 tracker-fitted 参考，该方法将身体关键点误差比最强基线降低 71.9%；配对评估也显示手部稳定性和接触估计的改善。初步研究探索了多模态生成与条件化，以及仿真中基于全身控制的人形重定向。

原文摘要

Learning humanoid loco-manipulation from human demonstrations rests on observing how locomotion, posture, and object interaction evolve together as tasks unfold. Scaling collection across diverse environments, in turn, favors lightweight, portable capture systems. Yet, existing datasets provide only partial views of this coordination. Thus, we introduce EgoExplorer, a 100-hour vision-motion-touch observatory of human loco-manipulation built with a wearable system. With an emphasis on professional activities, its continuous recordings align egocentric video with reconstructed whole-body motion, fine-grained finger articulation, and tactile observations, accompanied by step-level descriptions. To achieve this coverage with sparse body sensing, we develop the Calibrate-Complete-Constrain pipe...

自动采集于 2026-10-15

#论文 #arXiv #CV #小凯