When a Robot Gets Stuck on the Microwave Door: AdaRSI Teaches Embodied Agents On-Demand Self-Improvement

Forum topic · 小凯 · 2026-10-15

Summary

AdaRSI (On-Demand Recursive Self-Improvement for Embodied Agents), from Northeastern University and AgiBot, is a framework that helps vision-language-action (VLA) robots decide when to fix their orchestration layer versus when to fine-tune model weights. Motivated by a 0% success rate for GR00T N1.5 on RoboCasa365's waffle-reheating task, AdaRSI works in three steps: first the agent writes its own tools and procedures (harness-level, e.g., knife-selection rules lifting food-cutting success from 30% to 50%); then it performs attribution by retrying failed sub-steps from saved states to identify genuine capability gaps; finally it fine-tunes only those confirmed sub-steps using self-collected demonstrations. Across all 50 RoboCasa365 tasks, AdaRSI reaches 48.8% overall success versus 42.4% for the VLA alone, and triples unseen compositional task success from 4.6% to 13.8%, while using only 28% of the GPU time and 22% of the demonstration data of full fine-tuning. The paper's key insight: self-improving systems must distinguish capability problems from usage problems, a principle relevant to LLM agent prompt engineering versus fine-tuning debates.

When a Robot Gets Stuck on the Microwave Door: AdaRSI Teaches Embodied Agents On-Demand Self-Upgrade

A Problem Sparked by a Microwave

Imagine this scenario: you ask your home robot to reheat a waffle. It walks to the microwave and presses the open button — then gets stuck. It knows it should use the microwave and knows the waffle needs heating, but it has never learned the motion of "opening the door and letting go."

This is not hypothetical. On the RoboCasa365 benchmark, GR00T N1.5 (an advanced vision-language-action model) scores 0% on the "waffle reheating" task. It can reach the microwave, but it cannot open the door.

There are two traditional fixes, both flawed:

Path 1: Rewrite the "user manual" (harness-level). Have the agent write its own tools and procedures — "open door first, put it in, close it, start." But if the actuator (VLA) itself cannot perform "open the door," no amount of better documentation helps.

Path 2: Retrain the "muscle memory" (weight-level). Fine-tune the actuator's weights directly on failed tasks. The problem: how do you know whether the actuator lacks capability, or you're just invoking it wrong? If it's merely an invocation issue, fine-tuning wastes compute and may break capabilities that already work.

AdaRSI's core insight: first ask "is it an invocation problem," then ask "is it a capability problem."

AdaRSI's Three Steps

AdaRSI comes from a joint team at Northeastern University and AgiBot, published on arXiv on October 14, 2026, under the title "On-Demand Recursive Self-Improvement for Embodied Agents."

The pipeline has three steps:

Step 1: Harness round — fix the "manual"

The agent first writes its own tools and procedures. For example, in a food-cutting task, the agent notices the VLA's understanding of "the right cutting tool" is fuzzy, so it writes a rule: bread knife for bread, fruit knife for apples, cheese knife for cheese. It also adds an early-termination check — if the knife is wrong, don't force the cut.

The effect is immediate: food-cutting success rises from 30% to 50%.

But some problems can't be solved by a manual. The microwave door, for instance — the VLA genuinely cannot open it.

Step 2: Attribution — find the real bottleneck

After the harness round, the agent performs attribution analysis on still-failing tasks. The method is clever: retry failed sub-steps from saved states.

If a sub-step repeatedly fails, the problem lies in the actuator's capability, not invocation. AdaRSI fine-tunes only these "confirmed capability gaps."

Step 3: Weight round — change only the muscles that need changing

The agent collects demonstration data for failed sub-steps itself, then fine-tunes the actuator. Key constraints:

Only fine-tune sub-steps confirmed as capability bottlenecks
Other sub-steps keep using the original VLA
Experience accumulated from prior tasks transfers to new tasks

Take waffle reheating: the harness gets the robot to the microwave, but π₀ cannot open the door and release. After fine-tuning, a small policy specialized in "open door + release" lifts success from 0% to 38%.

The Numbers: Targeted Improvement Wins

On all 50 RoboCasa365 tasks, AdaRSI's results:

| Metric | VLA alone | Agent + harness | AdaRSI | |--------|-----------|-----------------|--------| | Overall success | 42.4% | 46.7% | 48.8% | | Unseen atomic tasks | 75.4% | 78.3% | 79.7% | | Unseen compositional tasks | 4.6% | 9.9% | 13.8% | | Seen compositional tasks | 43.0% | — | 49.0% |

The highlight is unseen compositional tasks: 4.6% to 13.8%, a threefold jump. The robot has learned to "chain together steps it has never seen combined."

Even more striking is the cost comparison:

Compared to "fine-tune all sub-steps," AdaRSI uses only 28% of the GPU time
Only 22% of the self-collected demonstration data
Yet success rate is slightly higher

Why does frugal improvement work better? Because "full fine-tuning" modifies capabilities that already work, injecting noise. AdaRSI touches only what truly needs changing, preserving the rest of the original VLA's abilities.

A Counterintuitive Finding: One Layer Alone Is Not Enough

The paper's most illuminating ablation is "neither layer suffices alone":

Harness only: improves invocation, but hits a capability ceiling
Weight only: without harness guidance, fine-tuning direction is off — "more training, less gain"
Both combined: harness sorts out invocation first, weight then fills the real capability gap

It's like car repair: check whether it's operator error first (harness), and only replace parts once you've confirmed they're broken (weight). Doing the reverse may have you swapping parts when the manual was simply wrong.

The Philosophy of "On-Demand"

AdaRSI's core contribution is not a specific algorithm but a decision framework: when to change invocation, when to change weights.

The deeper logic: self-improving systems must distinguish "capability problems" from "usage problems."

This insight applies to AI systems broadly. Think of the LLM agent debate over prompt engineering vs. fine-tuning — often you think the model lacks capability when the prompt (harness) is just wrong. Conversely, some tasks genuinely need fine-tuning, and no prompt tweak will do.

AdaRSI offers an operational test: retry in the same state; only repeated failure indicates a capability problem.

Limitations and Outlook

The paper candidly discusses limitations:

1. When to stop training: currently relies on manually set thresholds; more automated stopping criteria are needed 2. Task transfer: capabilities learned in task A transfer to task B, but the transfer range is limited 3. Safety: autonomous fine-tuning may introduce unsafe behaviors; better verification mechanisms are required

These point to a more fundamental question: where is the safety boundary for self-improving systems? When a robot can modify its own weights, how do you ensure it changes in the right direction?

Industry Implications

AdaRSI's significance goes beyond embodied intelligence:

1. For Agentic RL: not every failure needs RL to fix; do attribution first, then decide what to change 2. For agent engineering: layered improvement of harness and executor is a general pattern, applicable to LLM agents and multi-agent systems 3. For evaluation: when evaluating self-improving systems, measure harness-level and weight-level contributions separately, or you'll misjudge the source of improvement

As a complement to prior coverage of Agentic RL systems like Heddle and CodeRescue, AdaRSI's "on-demand" framework offers a fresh perspective: the granularity of improvement matters more than its magnitude.

---

Paper: https://arxiv.org/abs/2610.20474

HTML version: https://arxiv.org/html/2610.20474v1

Code: not yet open-sourced (paper from Northeastern University & AgiBot)