Paper Overview
Field: CV Authors: Heng Fang, Zehui Guo, Yuehan Zhang, Mingyang Du, Dingkang Liang Published: 2026-11-09 arXiv: 2611.12885
Summary (translated from Chinese)
Trained action policies write manipulation skills into network weights, tightly binding what is learned to the training distribution. In-context learning (ICL) avoids this coupling but has never matched trained policies—because a demonstration recorded in a different scene provides no feedback from the world the robot currently faces. The authors observe that what an agent learns in context is semantic information—task steps and object roles—rather than visual appearance, so experience gathered in one world can transfer to another.
Based on this insight, they propose the World In-Context Learning (WICL) paradigm: a frozen model starts from a demonstration, rehearses the task in a world such as a simulator, and writes the feedback into its context rather than its weights. They built RoboScript, modular infrastructure that implements WICL. From a single demonstration, RoboScript learns each task through rehearsal in simulation, freezes the acquired knowledge as a task memory, and deploys it in new scenes—a data isolation mechanism ensures every deployment input is obtainable on a real robot.
Results
- 83.4% average success rate across all 50 RoboTwin 2.0 tasks, in both clean and randomized settings
- Uses one demonstration per task and zero gradient updates
- Outperforms the strongest trained action policy by more than 20 percentage points
- Generalizes across visual distributions; self-improves through reflection; scales with foundation models
- Transfers to a physical dual-arm robot via cross-embodiment memory adaptation
Conclusion
When the world enters the loop, robots can learn manipulation tasks in context, matching learning-in-weights.
Code: https://github.com/ArcheOne-AI/RoboScript
Original Abstract (excerpt)
> Trained action policies write manipulation skill into network weights, tying what is learned to the distribution it is learned from. In-context learning (ICL) avoids this coupling but has not matched trained policies, because a demonstration recorded in another scene carries no feedback from the world the robot currently faces. We propose World In-Context Learning (WICL), a paradigm in which a frozen model starts from a demonstration, rehearses the task in a world such as a simulator, and writes the feedback into its context rather than its weights.
--- *Auto-collected on 2026-11-11*