Summary
RV-ICL (Recursive Video In-Context Learning) is a training-free method from Wenrui Bao, Xinxin Liu, Bingxin Xu, and Yuzhang Shang (arXiv 2610.06843) that lets LLM agents orchestrating frozen vision-language-action (VLA) policies learn from demonstration videos. Instead of text memory that records what the agent did, RV-ICL converts a single demonstration video into a hierarchical structure built from sub-events such as grasps and releases, with levels refining from whole-task keyframes to phases, moments, and short video clips. The agent reads coarse levels before planning and re-enters the hierarchy during execution, loading only clips relevant to the current subgoal. This preserves contact details that fixed keyframes lose while avoiding the cost of full-video context. Built on RPentomino-style infrastructure, RV-ICL raises LIBERO-PRO success from 92.6% to 96.5% and LIBERO-Plus from 86.7% to 95.8%, using only one demonstration per task.
Paper Overview
- Field: Robotics
- Authors: Wenrui Bao, Xinxin Liu, Bingxin Xu, Yuzhang Shang
- arXiv: 2610.06843
Introduction
LLM agents that orchestrate frozen vision-language-action (VLA) policies improve across episodes via text memory — but text memory records *what the agent did*, not *how the task should be done*. Demonstration videos show the how, yet they fit poorly in an agent's context: full videos slow every episode, fixed keyframes lose the contact details that decide whether a grasp succeeds, and the agent's needs shift from task structure during planning to frames near each contact point during execution.Method: RV-ICL
Recursive Video In-Context Learning (RV-ICL) is a training-free approach that turns a demonstration video into a hierarchy the agent navigates rather than a prompt it passively receives.
- The hierarchy is built from the demonstration's sub-events (e.g., grasp, release).
- Levels grow finer: keyframes of the whole task → phases → moments → short video clips.
- The hierarchy is exposed to the agent through read-only tools.
- The agent reads the coarse levels before planning, then re-enters the hierarchy during execution whenever a step needs more detail, loading only the clips relevant to the current subgoal.
- Only one demonstration per task is required.
Results
Built on RPent, RV-ICL achieves:
| Benchmark | Baseline | With RV-ICL |
|---|---|---|
| LIBERO-PRO | 92.6% | 96.5% |
| LIBERO-Plus | 86.7% | 95.8% |
Original Abstract
> RV-ICL turns a demonstration video into a hierarchy the agent navigates rather than a prompt it receives. Built on sub-events like grasps and releases, levels grow finer from keyframes to phases, moments and clips. One demonstration per task is enough, raising success from 92.6% to 96.5% on LIBERO-PRO.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178635681