Overview
- Field: Machine Learning
- Authors: Michal Korniak, Marco Bagatella, Ralf Römer, Michal Nauman, Yarden As, Pranav Vignesh, Manan Agarwal, Leander Diaz-Bone, Piotr Zalewski, Mikołaj Czarnecki, Lenart Treven, Shashwat Saxena, Sreyas Venkataraman, Sarvesh Patil, Zhiyuan Zhou, Daniel Marta, Oier Mees, Nur Muhammad Mahi Shafiullah, Carmelo Sferrazza, Bhavya Sukhija, Angela P. Schoellig, Andreas Krause, Max Simchowitz, Andrew Wagenmaker
- Published: 2026-11-04
- arXiv: 2611.06905
- Generalist policies can improve online across diverse tasks.
- With the right recipe, improvements generalize to tasks not covered during online training.
- The conditions for generalization are characterized: both the training task distribution and the use of value-based RL are critical.
- To the authors' knowledge, this is among the first evidence of online generalist improvement of robot foundation models.
Summary
Generalist robot policies allow a single, large model to learn many tasks from a vast pre-training corpus. While these models exhibit strong zero-shot generalization, they typically require an adaptation phase to reach sufficient performance in deployment. Iterated self-improvement via reinforcement learning (RL) has shown promise for this adaptation, but such methods have largely been explored in single-task settings—aimed at improving a narrow deployment task—in stark contrast to the multi-task capabilities of the base policy.
This work asks whether generalist robot policies can achieve multi-task, generalist improvement in deployment, targeting many diverse tasks simultaneously, and what algorithmic recipe enables it. The authors first develop structured evaluations of generalist improvement capability, systematically testing algorithmic and engineering design choices around value estimation, policy extraction, and resource allocation.