[论文] Computer-Use Agents Are Embodied Generalists

论文概要

研究领域: ML 作者: Pranjal Aggarwal, Sean Welleck 发布时间: 2026-11-06 arXiv: 2611.09847

中文摘要

在AI的许多领域，通用模型已日益取代手工方法和专用架构。本文认为具身AI正处于类似转变的前夜。核心主张是：具身任务可以被表述为与数字软件的交互。具体地，作者构建了将智能体对物理环境的视图渲染到屏幕、并提供在其中行动的控制接口的软件。计算机使用智能体（CUA）可以通过操作普通软件相同的截图、键盘和鼠标动作来执行具身任务。使用单一CUA设计、仅更换软件，作者的作品在四个ECCV研讨会竞赛中获得第一、第二、第三和第六名奖品，且无任何竞赛特定的智能体调优。这些结果表明CUA在软件上学到的技能——如视觉定位、精确点击和探索——已经能迁移到具身任务，使CUA在具身AI中的适用范围远超先前认知。因此作者主张计算机使用智能体可以成为具身通才，并通过遵循使CUA通用的训练策略，具身AI也能收敛到单一通用模型，开启通用具身AI的新时代。

原文摘要

In many areas of AI, general-purpose models have increasingly replaced handcrafted methods and specialized architectures. We argue that embodied AI is poised for a similar shift. Our key claim is that embodied tasks can be formulated as interaction with digital software. Specifically, we build software that renders an agent's view of a physical environment on a screen and provides controls for acting in it. A computer-use agent (CUA) can then perform embodied tasks through the same screenshots and keyboard and mouse actions that it uses to operate ordinary software. Using a single CUA design and changing only the software, our entries win first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions, without any competition-specific agent tuning. These results sugg...

#论文 #arXiv #ML #小凯