[论文] Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demo...

论文概要

研究领域: CV
作者: Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung, Yongjun You, H. Jin Kim, Jia-Bin Huang, Furong Huang, Youngseok Jang, Seungjae Lee
发布时间: 2026-10-08
arXiv: 2610.12470

中文摘要

从单个人类视频中学习灵巧操作，为替代昂贵的机器人演示提供了一条可行路径。然而近期方法大多以模仿演示动作为主，这种严格的动作匹配往往限制了模型对视频中未出现的初始物体位姿、目标位姿和抓取方式的泛化能力。另一方面，通过强化学习（RL）发现策略虽可广泛泛化，但缺乏先验引导时在复杂多阶段任务的高维探索中举步维艰。为解决这对耦合的泛化与探索难题，我们提出 Dex-One2Many——一个从单个人类视频学习可泛化灵巧操作策略的 real-to-sim-to-real 框架。核心洞见是将视频抽象为时序场景图来引导 RL：场景图作为生成式约束用于采样多样化重置状态，并为每阶段提供稠密奖励。由于场景图约束的是物体间关系而非精确位姿，重置状态可覆盖超出视频范围的物体位姿与抓取方式；同时稠密奖励使探索短而有引导。模型完全在仿真中训练，零样本迁移到真实多指机械手。在五个工具使用与操作任务上，Dex-One2Many 在已见配置下超过基线 6.5%，未见场景中这一优势扩大至 71%。

原文摘要

While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human video. Our key insight is to abstract the video into sequential scene g...

自动采集于 2026-10-10

#论文 #arXiv #CV #小凯