RoboPocket: Improving Robot Policies Instantly with Your Phone

Forum topic · 小凯 · 2026-03-07

Summary

RoboPocket is a robotics research paper by Junjie Fang, Wendi Chen, Han Xue, and colleagues from a team including Chuan Wen and Cewu Lu, published on arXiv (2603.05504) in robotics (cs.RO), AI (cs.AI), and machine learning (cs.LG). The work addresses a core bottleneck in imitation learning: the efficiency of data collection. Handheld interfaces, such as phones, enable scalable in-the-wild demonstration gathering, but they typically operate open-loop—operators collect demonstrations without knowing where the current policy fails, resulting in inefficient coverage of critical state distributions. In contrast, interactive approaches like DAgger can mitigate covariate shift by targeting policy weaknesses, but they depend on physical setups that limit scalability. RoboPocket proposes to combine the scalability of phone-based handheld data collection with closed-loop, policy-aware guidance, allowing operators to improve robot policies instantly using their phone. The abstract was auto-collected from arXiv; the full method details and experimental results are available in the original paper.

RoboPocket: Improve Robot Policies Instantly with Your Phone

Authors: Junjie Fang, Wendi Chen, Han Xue, Fangyuan Zhou, Tian Le, Yi Wang, Yuting Zhang, Jun Lv, Chuan Wen, Cewu Lu

arXiv: 2603.05504 — cs.RO, cs.AI, cs.LG

Abstract

Scaling imitation learning is fundamentally constrained by the efficiency of data collection. While handheld interfaces have emerged as a scalable solution for in-the-wild data acquisition, they predominantly operate in an open-loop manner: operators blindly collect demonstrations without knowing the underlying policy's weaknesses, leading to inefficient coverage of critical state distributions. Conversely, interactive methods like DAgger effectively address covariate shift but rely on physical infrastructure, limiting their scalability.

RoboPocket proposes combining the scalability of phone-based handheld data collection with policy-aware, closed-loop guidance so that robot policies can be improved instantly using a phone.

Key Points

Problem: imitation learning is bottlenecked by how efficiently demonstration data can be collected.
Existing handheld interfaces scale well but collect data open-loop, blind to the policy's failure modes.
Interactive methods such as DAgger target policy weaknesses but require physical setups that limit scalability.
RoboPocket aims to close this gap by letting operators use their phone to both collect data and target the current policy's weaknesses.