RPG: Guided Self-Improvement for Embodied Robots Without Weight Updates

Forum topic · 小凯 · 2026-10-03

Summary

Researchers present Reconstruct, Practice, Go Real (RPG), a framework enabling autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities in an offline dataset, then constructs related practice tasks in simulation. During practice, it diagnoses failures using execution feedback, privileged simulator state, and dataset videos, developing new reusable symbolic skills, refining existing skills, and revising the system prompt. Candidate changes are tested cross-task before being retained. At test time, a multimodal LLM coordinates perception and robot control using the generated system prompt and skill library. Across 22 manipulation tasks with held-out initializations, RPG raised success rates from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming baselines including ASPIRE (75.5%) and CaP-Agent0 (60.0%). After general calibration and hardware adaptation, the frozen system succeeded in all 30 physical experiments.

Overview

Field: Machine Learning (ML) Authors: Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi Published: 2026-10-01 arXiv: 2610.02204

Abstract

Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights.

How RPG Works

Reconstruct: RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation.
Practice: During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures. It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses.
Cross-task evaluation: Individual candidate changes and merged revisions are tested before being retained for reuse.
Go Real: At test time, a multimodal LLM coordinates perception and robot control using the generated system prompt and skill library.

Results

Across 22 manipulation tasks with held-out initializations, RPG improved task success from 28.6% after the first practice round to 95.0% after 15 rounds.
RPG outperformed all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 (60.0%).
After general calibration and hardware adaptation, the frozen system succeeded in all 30 physical experiments.

Significance

RPG demonstrates that embodied systems can self-improve through guided simulation practice and prompt/skill library refinement, achieving strong real-world transfer without any weight updates.

---

Source: arXiv:2610.02204