World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays (REGEN)

Forum topic · 小凯 · 2026-06-27

Summary

Researchers propose REGEN (Recurrent Generative Replay), a continual imitation learning framework for robots built on World Action Models (WAMs). Beyond predicting robot actions, WAMs can generate future visual observations. REGEN exploits this generative capability to synthesize pseudo-replay trajectories, allowing a robot policy to rehearse previously learned tasks without storing the original human demonstrations. During continual adaptation, REGEN recursively queries the WAM to produce these synthetic trajectories, which are used to review past tasks while learning new ones. This mitigates catastrophic forgetting in continual learning settings, enabling robots to keep acquiring new skills while maintaining performance on earlier ones. The authors—Manish Kumar Govind, Dominick Reilly, and Smit Patel—report significant improvements in continual learning performance across a variety of robot manipulation tasks. The paper is available on arXiv as 2606.27374 (June 2026). The work spans computer vision and robotics, addressing a key bottleneck in lifelong robot learning: retaining old skills under storage constraints.

Overview

Field: Computer Vision / Robotics
Authors: Manish Kumar Govind, Dominick Reilly, Smit Patel
Published: 2026-06-27
arXiv: 2606.27374

Abstract (from the paper)

Going beyond predicting robot actions, World Action Models (WAMs) can also generate future visual observations. We build on this generative capability to propose Recurrent Generative Replay (REGEN), a continual imitation learning framework that synthesizes pseudo-replay trajectories, enabling a robot policy to rehearse previously learned tasks without storing their original human demonstrations. During continual adaptation, REGEN recursively queries the WAM to synthesize pseudo-replay trajectories, which are used to review past tasks while learning new ones. This approach mitigates catastrophic forgetting in continual learning scenarios, allowing robots to continually acquire new skills while retaining performance on previously learned tasks. The authors demonstrate REGEN's effectiveness across various robot manipulation tasks, showing significant improvements in continual learning performance.

Key Points

World Action Models can generate future visual observations, not just robot actions.
REGEN uses this generative capability to create pseudo-replay trajectories for rehearsal.
No need to store original human demonstrations of past tasks.
Recurrent querying of the WAM during continual adaptation reduces catastrophic forgetting.
Validated on multiple robot manipulation tasks with notable gains over continual learning baselines.

Full paper: https://arxiv.org/abs/2606.27374