Summary
WebHarbor is a suite of 153 executable, dockerized website environments that reconstruct the visual interfaces, content, and functional workflows of real-world web services under deterministic control. It addresses the key limitations of GUI web agent research: live-website evaluation suffers from changing content, access restrictions, and poor coverage of authenticated workflows, while existing synthetic training environments lack functional depth and visual fidelity. WebHarbor is built through task-driven human-agent collaboration in which environments expand their functional scope as new tasks arrive, coding agents implement the changes, and human contributors audit visual fidelity, functional correctness, and task validity. The released suite contains 3,518 tasks covering multi-step and post-authentication workflows that prior work rarely covers. Beyond reliable evaluation, WebHarbor serves as a high-fidelity synthetic web environment for low-cost reinforcement learning. Qwen 4B-scale GUI agents trained with RL in WebHarbor generalize to unseen tasks and compete with leading open-source GUI web agents across four benchmarks. Code and data are open-sourced at https://github.com/aiming-lab/WebHarbor (arXiv: 2611.03634).
论文概要
研究领域: NLP
发布时间: 2026-11-02
arXiv: 2611.03634
Overview
GUI web agents require realistic and interactive environments that remain stable during both evaluation and training. Live website evaluation suffers from changing content, access restrictions, and limited coverage of authenticated workflows, while existing synthetic training environments often lack functional depth and visual fidelity.
What WebHarbor Provides
- 153 executable, dockerized website environments that faithfully reconstruct the visual interfaces, content, and functional workflows of real-world web services under deterministic control.
- 3,518 tasks covering multi-step and post-authentication workflows — areas that existing work frequently fails to cover.
Construction Methodology
WebHarbor is built through task-driven human-agent collaboration:
1. Environments evolve as new tasks expand their functional scope.
2. Coding agents implement the required changes.
3. Human contributors audit visual fidelity, functional correctness, and task validity.
Reinforcement Learning Results
Beyond reliable evaluation, WebHarbor provides a high-fidelity synthetic web environment for low-cost RL:
- A Qwen 4B-scale GUI agent trained with RL in WebHarbor generalizes to unseen tasks.
- It is competitive with leading open-source GUI web agents across four benchmarks.
Resources
- Paper: arXiv:2611.03634
- Code and data: https://github.com/aiming-lab/WebHarbor
---
*Auto-collected on 2026-11-04*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178636722