Paper Overview
- Field: Computer Vision / Robotics
- Authors: Zhao-Heng Yin, Guanya Shi, Pieter Abbeel, C. Karen Liu
- Published: 2026-06-11
- arXiv: 2606.13677
- Reinterprets dexterous articulated tool manipulation as an animation problem
- Coarse-to-fine pipeline combining procedurally generated grasp keyframes, motion planning, and reinforcement learning
- Highly automated data generation with minimal manual annotation
- Zero-shot sim-to-real transfer demonstrated across tools of varying scales and joint types
Abstract (translated)
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies.
The authors present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning.
The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (less than one minute per tool). Across articulated tools spanning four different scales and joint types, Mana achieves zero-shot sim-to-real transfer of both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.