[论文] [论文] Profiling a Frontier Agent with EBench: From Direct Control to Ha...

论文概要

研究领域: ML
作者: Yifei Yao, Yude Zou, Guanbo Jiao, Xing Gao, Ning Gao et al.
发布时间: 2026-10-26
arXiv: 2610.35832

中文摘要

前沿多模态大模型（MLLM）无需机器人专项训练即可控制机器人，但它们的能力与后训练 VLA/WAM 策略相比如何、在操作需求上如何互补，仍缺乏系统评估。本报告在 EBench 上评估 GPT-6-Astra：它作为智能体核心，在与后训练策略完全相同的视觉与本体感知观测协议下经 harness 控制机器人。研究两种配置：直接控制（直接生成路点指令）与 harness+策略（还可调用后训练 VLA/WAM 执行动作）。直接控制下整体表现与后训练策略相当：短程任务领先，跨泛化维度方差最小；但长程任务、桌面操作与高精度接触富集任务落后。harness+策略 下，调用后训练策略结合双方互补优势，整体成功率提升到最强后训练策略水平，同时大幅降低 token 成本；但仍低于各组件单独使用的任务级上界，原因是策略调用欠优、按固定指令执行缺乏子目标、智能体接管时抓握被破坏。本研究刻画了前沿 MLLM 智能体的能力边界，为改进 harness 设计指明方向。

原文摘要

Frontier multimodal large language models (MLLMs) can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action (VLA) and world action model (WAM) policies across manipulation demands remains to be thoroughly evaluated. In this report, we evaluate GPT-6-Astra on EBench as the core of an agent that controls a robot through an agent harness under the same visual and proprioceptive observation protocol as post-trained policies. We study two agent configurations: direct control, in which the agent generates waypoint commands directly, and harness with policy, in which it can also invoke a post-trained VLA or WAM policy for action execution. Under direct control, the agent achieves competitive overall performance relative to post-trained policies. It leads on short-horizon tasks and exhibits the least performance variation across generalization dimensions, whereas it underperforms post-trained policies on long-horizon tasks, tabletop manipulation, and high-precision contact-rich tasks. In the harness with policy, invoking a post-trained VLA or WAM combines the complementary strengths of both components, raising overall task success to the level of the strongest compared post-trained policies while substantially reducing token cost. However, performance remains below the per-task upper bound of the standalone components due to suboptimal policy invocation, fixed-instruction execution without subgoals, and grasp disruption when the agent resumes control. Overall, our study characterizes the capability boundary of a frontier MLLM agent and suggests directions for improving future agent harness design. Project page: https://internrobotics.github.io/EBench-Astra-Tech-Report/

自动采集于 2026-10-28

#论文 #arXiv #ML #小凯