Recursive Agent Optimization (RAO): RL for Training Recursive LLM Agents

Forum topic · 小凯 · 2026-05-11

Summary

Recursive Agent Optimization (RAO) is a reinforcement learning approach for training recursive agents—LLM-based agents that can spawn new instantiations of themselves and delegate sub-tasks recursively. Recursive agents implement an inference-time scaling algorithm, enabling divide-and-conquer over long contexts and more difficult problems. RAO trains models to best exploit this recursive inference by teaching agents when and how to delegate and communicate. The authors report that agents trained with RAO achieve better training efficiency, can scale to tasks exceeding the model's context window, generalize to substantially harder tasks than those seen during training, and can reduce wall-clock time compared to single-agent systems. The paper is authored by Apurva Gandhi, Satyaki Chakraborty, Xiangjun Wang, Aviral Kumar, and Graham Neubig, posted to arXiv (2605.06639) on May 7, 2026, in the NLP domain.

Paper Overview

Field: NLP Authors: Apurva Gandhi, Satyaki Chakraborty, Xiangjun Wang, Aviral Kumar, Graham Neubig Published: 2026-05-07 arXiv: 2605.06639

Abstract

We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and delegate sub-tasks to new instantiations of themselves recursively. Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts and generalize to more difficult problems via divide-and-conquer. RAO provides a method to train models to best take advantage of such recursive inference, teaching agents when and how to delegate and communicate. We find that recursive agents trained in this way enjoy better training efficiency, can scale to tasks that go beyond the model's context window, generalize to tasks much harder than the ones the agent was trained on, and can enjoy reduced wall-clock time compared to single-agent systems.

Key Takeaways

RAO is a reinforcement learning method for training recursive agents that delegate sub-tasks to spawned copies of themselves.
Recursive delegation acts as an inference-time scaling algorithm based on divide-and-conquer.
Reported benefits: improved training efficiency, scaling beyond the context window, generalization to harder tasks, and reduced wall-clock time versus single-agent setups.

--- *Auto-collected on 2026-05-11*