Overview
Field: Computer Vision / Embodied AI Authors: Yunsong Zhou, Hangxu Liu, Xuekun Jiang Published: 2025-04-10 arXiv: 2504.07903
Summary
Robotic manipulation with deformable objects represents a data-intensive regime in embodied learning, where shape, contact, and topology co-evolve in ways that far exceed the variability of rigid objects. Although simulation promises relief from the cost of real-world data collection, prevailing sim-to-real pipelines remain rooted in rigid-body abstractions, producing mismatched geometry, fragile soft dynamics, and motion primitives poorly suited for cloth interaction. The authors posit that simulation fails not for being synthetic, but for being ungrounded.
To address this, the paper introduces SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world. Given limited demonstrations, the system:
1. Digitizes scenes into metric-consistent digital twins. 2. Calibrates deformable dynamics through elasticity modeling. 3. Scales behaviors via diffusion-based trajectory generation combined with quality filtering.
This pipeline converts sparse observations into scalable synthetic supervision with near-demonstration fidelity.
Results
- Policies trained only on synthetic data achieve performance comparable to real-data baselines at an equivalent data ratio of 1:15 (1 real sample equivalent to 15 synthetic ones).
- Real-world deployment achieves a 90% zero-shot success rate.
- A 50% generalization gain in real-world settings.
Conclusion
The results validate physically aligned simulation as a practical path toward scalable supervision and data-efficient policy learning for deformable object manipulation.