Paper Overview
- Field: NLP / Multimodal GUI Agents
- Authors: Venus Team, Zhuohan Cai, Haoxing Chen
- Released: 2026-09-03
- arXiv: 2509.00008
- Unified closed-loop reasoning-action framework spanning mobile, web, and desktop.
- Large-scale environment coverage: 170+ multilingual apps plus native desktop OS.
- Function-grounded instruction generation via a deep-research pipeline.
- Reliable RL rewards through multi-model voting and visual keypoint verification.
- Open-source foundation aimed at general, verifiable, self-reflective GUI agents.
Abstract
Multimodal GUI agents have emerged as a promising paradigm for digital task automation, yet transitioning from benchmark-oriented models to dependable real-world applications remains challenging due to limited environment coverage, brittle task construction, and unreliable reward verification. In this work, we present UI-Venus-2, a general-purpose foundation GUI agent designed to operate across mobile, web, and desktop environments through a unified closed-loop reasoning-action framework.
To bridge the gap toward practical deployment, the authors jointly scale three critical dimensions:
1. Environments — expanding coverage to more than 170 multilingual mobile apps and native desktop operating systems; 2. Tasks — employing a deep-research pipeline for function-grounded instruction generation; 3. Verification — using trace-level and sample-level evaluators with visual keypoints and multi-model voting to ensure reliable RL training signals.
Additionally, safety-aware mechanisms are integrated to ensure controlled execution of critical actions.
Key Takeaways
*Auto-collected on 2026-09-03.*