[论文] PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Re...

论文概要

研究领域: ML 作者: Kewei Zhang, Juncheng Ma, Guodong Lv, Duomin Wang, Feixiang Ruan, Ruiyang Si et al. 发布时间: 2026-11-03 arXiv: 2611.05320

中文摘要

遥操作仍是采集机器人演示的默认方式，但每条 episode 都消耗机器人和操作员时间，每个新任务又要从零开始。我们追问：编程智能体能否接管遥操作员的工作？在我们的 DataFarm 中，智能体收到一份数据请求和机器人在目标场景中的一条标定 episode，然后在仿真中重建任务：它编写场景、专家控制器和成功判定器。仿真器提供精确动作，一个以真实图像为条件的视频模型把每次 rollout 渲染成照片级视频，并一致地生成所有相机视角。产出是「real-ready」数据：与机器人动作同步标注的相机视角，可直接在其上训练策略并部署到真实机器人。独立的裁判在准入前认证专家与每条渲染 episode。在五个真机任务上，仅在农场数据上训练的策略与用同等数量遥操作 episode 训练的策略相当（100 次试验中 93 次成功对 84 次），在未见过的桌布和彩色灯光下，农场数据训练的策略保持 87 次成功，而遥操作训练的策略跌至 46。适配一个任务完全自动，平均每次任务仅花 21 美元 API 费用，外加 5 美元渲染契约；之后每条 episode 的仿真、渲染与审查平均仅 0.80 美元，不再需要机器人或操作员。一个任务的一小时三视角数据平均成本 107 美元，略低于中国 117 美元的遥操作报价，远低于北美仅人力就 290 美元的水平。

原文摘要

Teleoperation remains the default way to collect robot demonstrations, but every episode costs robot and operator time, and every new task starts from scratch. We ask whether a coding agent can take over the teleoperator's job. In our DataFarm, the agent receives a data request and one calibration episode of the robot in the target scene and rebuilds the task in simulation: it writes the scene, an expert controller, and a success judge. The simulator supplies exact actions, and a video model conditioned on real images renders each rollout as photographic video, generating all camera views together consistently. The output is real-ready data: synchronized camera views labelled with the robot's actions, on which a policy can be trained directly and deployed on the real robot. Independent judges certify the expert and every rendered episode before admission. On five real-robot tasks, policies trained only on farm data match policies trained on as many teleoperated episodes (93 vs. 84 successes in 100 trials), and under an unseen tablecloth and coloured light they keep 87 successes where the teleoperation-trained policies fall to 46. Adapting a task is fully automatic and costs \$21 of API usage on average, paid once per task, plus \$5 for a render contract; every further episode costs \$0.80 on average for simulation, rendering, and review, with no robot or operator. One hour of three-view data for a task costs \$107 on average, these costs included, slightly below a quote of \$117 for teleoperation in China and below \$290 in labour alone in North America.

---

*自动采集于 2026-11-05*

#论文 #arXiv #AI #小凯