Mana: Dexterous Manipulation of Articulated Tools — A Sim-to-Real Framework from Computer Animation

Forum topic · 小凯 · 2026-06-14

Summary

Researchers Zhao-Heng Yin, Guanya Shi, and Pieter Abbeel present Mana (Manipulation Animator), a general sim-to-real framework for dexterous manipulation of articulated tools, described in arXiv paper 2506.10668. Articulated tool manipulation is challenging because it requires coordinating internal degrees of freedom with contact-rich interactions; prior work has mostly addressed rigid objects. Mana reinterprets dexterous manipulation as an animation problem: inspired by computer animation, it uses a coarse-to-fine pipeline that converts procedurally generated grasp keyframes into manipulation trajectories via motion planning and reinforcement learning. Data generation is largely automatic, requiring only a few mouse clicks (under one minute per tool) to specify functional affordances. Experiments across four articulated tools of varying scales and joint types show that Mana achieves zero-shot sim-to-real transfer in both grasping and in-hand manipulation, offering a scalable approach to functional articulated tool use. This post summarizes the paper's abstract for the robotics and computer vision community.

Paper Overview

Field: CV / Robotics Authors: Zhao-Heng Yin, Guanya Shi, Pieter Abbeel Published: 2025-06-13 arXiv: 2506.10668

Abstract (Translated)

Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies.

The authors present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning.

The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances — less than one minute per tool. Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer in both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.

Key Takeaways

Reframes dexterous articulated manipulation as an "animation" problem, borrowing ideas from computer animation
Coarse-to-fine pipeline: procedural grasp keyframes → motion planning + reinforcement learning → full manipulation trajectories
Highly automated data generation with minimal human input (a few clicks to label functional affordances)
Zero-shot sim-to-real transfer demonstrated on four articulated tools with varying scales and joint types
Covers both functional grasping and in-hand manipulation