RAO: Recursive Agent Optimization — Teaching AI to Clone Itself

> When a smart person is asked to remember a hundred things at once, they choose to — create ninety-nine copies of themselves.

The post opens with an analogy: a CEO facing a complex project with hundreds of thousands of variables (design, traffic, energy, fire safety, resident data) has only a finite desk — like an LLM's context window. Overloaded, details get corrupted and lost. The solution is management wisdom: split the project across departments, each with its own workspace, then integrate results. This is the core intuition behind recursive agents.

Yet most LLM agents today remain "lone heroes" — one model, one context window, one long reasoning chain. On 2026-05-08, a joint CMU + Amazon AGI Labs team published Recursive Agent Optimization (RAO), asking: what if agents could *learn by training* when to split tasks, how to split them, and how to aggregate results?

Paper details

| Item | Content | |------|---------| | Title | Recursive Agent Optimization | | Authors | Apurva Gandhi, Satyaki Chakraborty, Xiangjun Wang, Aviral Kumar, Graham Neubig | | Institutions | CMU + Amazon AGI Labs | | arXiv | 2605.06639 | | Released | 2026-05-08 | | Core contribution | First RL method specifically for training recursive agents |

Part 1: Why single agents break down

Today's agents work step by step: generate a thought, call a tool, append results to the context window — like writing on a shrinking whiteboard. Context-window limits are a hard ceiling: on TEXTCRAFT-SYNTH the single agent is allowed 200 steps; on DEEPDIVE, 100 steps — but as history balloons, the model loses direction, loops on irrelevant ideas, and forgets early constraints.

Deeper problem: train/inference misalignment. Existing methods (RLHF, DPO, online RL variants) optimize single steps or short chains, not how to plan a 100-step task. Agents shine on short tasks but stall on long ones.

Part 2: The recursive agent's three superpowers

When a task is too complex, the agent creates a sub-agent — a copy of itself with a fresh, independent context window — delegates a subtask, and later receives the result. RAO is the training method that teaches models to use this effectively.

1. Expanded effective working memory — sub-agents start with clean windows carrying only relevant instructions. Recursive setup: root agent 25 steps + each sub-agent 25 steps, vs. 200-step single-agent baselines. 2. Natural divide-and-conquer — parents split and merge; children solve. Crucially, the splitting is *learned*, not hard-coded. 3. Concurrency reduces wall-clock time — multiple sub-agents run in parallel; the agent "delegates" smarter rather than "thinks" faster.

Part 3: RAO — training the model, not just scaffolding it

Key insight: "Inference-time scaffolding should not merely be designed around models; models should be trained to use them." Prior sub-agent work (Anthropic 2025, OpenAI 2026) built architectures but didn't train models to use delegation optimally. RAO is the first RL method for recursive agents, built on:

• AReAL asynchronous RL backend — sub-agents collect experience on parallel timelines, essential for recursive structure.
• CISPO-style objectives — consistency constraints across sampled trajectories let different subtask decompositions of the same parent task be compared, pushing the model toward better split strategies.

Training loop: (1) sampling — at each step the agent either continues itself or spawns a sub-agent; (2) credit assignment — final success/failure is propagated back through the recursion tree to each delegation decision; (3) policy update. The result is dynamic judgment of when, how finely, and how to split — not a fixed template.

Part 4: Experiments

• TEXTCRAFT-SYNTH (code synthesis): higher completion rates and markedly better training efficiency — recursion compresses effective problem complexity, shrinking search spaces so RL converges faster.
• OOLONG-REAL (long-context aggregation): different document subsets go to different sub-agents for local reasoning; the parent performs global aggregation — like analysts summarizing sections for an editor.
• DEEPDIVE (deep research): recursive agents generalize to tasks much harder than training ones; recursion depth can extend beyond what was seen during training.

Core data points:

• Better training efficiency (higher performance in fewer steps)
• Effective length beyond the native context window
• Generalization to harder-than-trained complexity
• Reduced wall-clock time via concurrent sub-agents
• 200/100 steps (single) vs. 25+25 (recursive): reorganized information flow, not more steps

Part 5: Implications and limitations

1. From bigger windows to smarter organization — context-window arms races (4K→1M→10M) miss that a clean 25-step window can beat a messy 200-step one; recursion is an information-architecture innovation. 2. A training paradigm shift — optimizing *meta-decisions* ("when to delegate") rather than individual steps; from micro-management training to strategic training. 3. Limitations: high training cost and complex credit assignment; likely diminishing returns beyond shallow recursion (the 25+25 config hints at this); delegation granularity is a hard-to-learn latent variable; generality on open-ended tasks (creative writing, emotional dialogue, legal argument) is untested. 4. Relation to prior work: Anthropic Subagents (2025) and OpenAI Subagents (2026) built architecture without optimal-use training; Context-Folding (Sun et al., 2025) compresses context where RAO *distributes* it; Recursive Language Models (Zhang et al., 2025) recurse within a model, while RAO recurses at the agent-system level. RAO's uniqueness: turning recursion from an architectural feature into a learnable policy.

Part 6: A thought experiment

Suppose you can clone yourself instantly, but each clone handles one task and cloning costs energy. For "write one email," don't clone. For "read 100 papers and write a survey in a day," spawn ten readers and one integrator. Now imagine the AI must learn by itself when to clone, how many, and how to integrate — that is RAO's core problem. The vision is not a "stronger" AI but one that "knows not to overreach": aware of its limits, skilled at delegation and integration. Agent training henceforth asks not only "how to take this step" but "who should take it."

References

1. Gandhi, A., Chakraborty, S., Wang, X., Kumar, A., & Neubig, G. (2026). *Recursive Agent Optimization*. arXiv:2605.06639. 2. Anthropic. (2025). *Subagents: Scaling Language Model Reasoning through Recursive Decomposition*. 3. OpenAI. (2026). *Subagents for System-2 Reasoning*. 4. Sun, Y., et al. (2025). *Context-Folding: Compressing Long Context for Efficient Inference*. 5. Zhang, L., et al. (2025). *Recursive Language Models: Iterative Refinement of Latent States*. 6. Feynman, R. P. (1965). *The Character of Physical Law*. MIT Press. 7. Asimov, I. (1951). *Foundation*. Gnome Press.