Paper Overview
Field: Machine Learning Authors: Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang Published: 2026-06-10 arXiv: 2606.12372
Summary
Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration. This incurs high labor cost and limits real-world scalability.
To address this, the authors propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators.
Method Components
1. Future-conditioned action-value estimation: predicts the latent consequence of the current action and evaluates its induced value, providing a more stable progress signal. 2. Temporal value-risk critic: aggregates recent value dynamics and triggers intervention when estimated value shows persistent stagnation or degradation. 3. Memory-based recovery: when intervention is needed, UniIntervene retrieves high-value recovery targets from memory of past intervention episodes and produces executable corrective actions via a goal-conditioned recovery policy.
Through this design, UniIntervene transforms intervention from passive human correction into a value-aware recovery process, enabling efficient real-world RL.
Results
Extensive experiments on diverse real-world manipulation tasks show that UniIntervene improves average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.
Original Abstract (excerpt)
> Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real-world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators.
*Auto-collected on 2026-06-12*