English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Computer-Use Agents Are Embodied Generalists: Paper Overview

Forum topic · 小凯 · 2026-11-09

Summary

A new arXiv paper (2611.09847) by Pranjal Aggarwal and Sean Welleck argues that embodied AI is poised for a shift toward general-purpose models, similar to what happened in other AI fields. The core idea is that embodied tasks can be formulated as interaction with digital software: the authors built software that renders an agent's view of a physical environment on screen and provides control interfaces, allowing a computer-use agent (CUA) to perform embodied tasks via the same screenshots, keyboard, and mouse actions it uses for ordinary software. Using a single CUA design and changing only the software, their entries won first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions without any competition-specific agent tuning. The results suggest that skills CUA models learn in software—visual grounding, precise clicking, and exploration—transfer to embodied tasks, indicating computer-use agents can serve as embodied generalists and pointing toward a single universal model for embodied AI.

Computer-Use Agents Are Embodied Generalists

  • Field: Machine Learning
  • Authors: Pranjal Aggarwal, Sean Welleck
  • Published: 2026-11-06
  • arXiv: 2611.09847
  • Key points

  • In many areas of AI, general-purpose models have replaced handcrafted methods and specialized architectures. The authors argue embodied AI is poised for a similar shift.
  • Core claim: embodied tasks can be formulated as interaction with digital software.
  • The authors build software that renders an agent's view of a physical environment on a screen and provides controls for acting within it.
  • A computer-use agent (CUA) can then perform embodied tasks using the same screenshots, keyboard, and mouse actions it uses to operate ordinary software.
  • Using a single CUA design and changing only the software, their entries won first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions — with no competition-specific agent tuning.
  • Skills CUA models learn on software, such as visual grounding, precise clicking, and exploration, appear to transfer to embodied tasks.
  • Conclusion: computer-use agents can act as embodied generalists; with training strategies that make CUAs general, embodied AI may converge toward a single universal model, opening a new era of general-purpose embodied AI.

Original abstract (excerpt)

> In many areas of AI, general-purpose models have increasingly replaced handcrafted methods and specialized architectures. We argue that embodied AI is poised for a similar shift. Our key claim is that embodied tasks can be formulated as interaction with digital software. Specifically, we build software that renders an agent's view of a physical environment on a screen and provides controls for acting in it. A computer-use agent (CUA) can then perform embodied tasks through the same screenshots and keyboard and mouse actions that it uses to operate ordinary software. Using a single CUA design and changing only the software, our entries win first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions, without any competition-specific agent tuning...

*Auto-collected on 2026-11-10.*

Tags

#machine-learning#embodied-ai#computer-use-agents#paper#arxiv#eccv#generalist-models

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636989