ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

[论文] ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

论文概要

研究领域: cs.LG, cs.AI, cs.CL, cs.CV
作者: Fei Tang, Zhiqiong Lu, Boxuan Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
发布时间: 2026-04-13
arXiv: 2604.11784

中文摘要

GUI智能体通过视觉界面而非程序化API驱动应用程序，通过点击、滑动和按键与任意软件交互，达到基于CLI的智能体无法触及的长尾应用。然而，该领域的进展瓶颈不在于建模能力，而在于缺乏连贯的全栈基础设施。本文提出ClawGUI，一个开源框架，在一个统一平台内解决这三个差距。ClawGUI-RL提供首个开源GUI智能体RL基础设施；ClawGUI-Eval在6个基准和11个以上模型上强制执行标准化评估流程；ClawGUI-Agent通过12个以上聊天平台将训练好的智能体带到Android、HarmonyOS和iOS。端到端训练的ClawGUI-2B在MobileWorld GUI-Only上达到17.1%成功率。

原文摘要

GUI agents drive applications through their visual interfaces instead of programmatic APIs, interacting with arbitrary software via taps, swipes, and keystrokes, reaching a long tail of applications that CLI-based agents cannot. Yet progress in this area is bottlenecked less by modeling capacity than by the absence of a coherent full-stack infrastructure.

自动采集于 2026-04-15

#论文 #arXiv #AI #小凯