Vision2Web: A Hierarchical Benchmark for Visual Website Development

Forum topic · 小凯 · 2026-03-31

Summary

Vision2Web is a hierarchical benchmark introduced to systematically evaluate large language model coding agents on end-to-end visual website development. Built from real-world websites, it spans three levels of difficulty: static UI-to-code generation, interactive multi-page frontend reproduction, and long-horizon full-stack website development. The benchmark contains 193 tasks across 16 categories, 918 prototype images, and 1,255 test cases. For flexible, thorough, and reliable evaluation, the authors propose a workflow-based agentic verification paradigm combining two complementary components: a GUI agent verifier and a VLM-based judge. Evaluations of multiple vision-language models instantiated in different coding agent frameworks reveal significant performance gaps at all task levels, with even state-of-the-art models struggling on full-stack development tasks. The paper is available on arXiv as 2503.23730. This post summarizes the benchmark design, evaluation methodology, and key findings for researchers working on code generation and agentic evaluation.

Paper Overview

Research Area: Machine Learning Authors: Zehai He, Wenyi Hong, Zhen Yang Published: 2025-03-30 arXiv: 2503.23730

Abstract

Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited. To address this gap, the authors introduce Vision2Web, a hierarchical benchmark for visual website development, spanning from static UI-to-code generation, interactive multi-page frontend reproduction, to long-horizon full-stack website development.

The benchmark is constructed from real-world websites and comprises a total of 193 tasks across 16 categories, with 918 prototype images and 1,255 test cases. To support flexible, thorough, and reliable evaluation, the authors propose a workflow-based agentic verification paradigm built on two complementary components: a GUI agent verifier and a VLM-based judge.

Evaluations of multiple vision-language models instantiated under different coding agent frameworks reveal that significant performance gaps remain at all task levels, and state-of-the-art models still struggle with full-stack development.

Key Highlights

Hierarchical benchmark covering three levels: static UI-to-code, multi-page interactive frontend, and long-horizon full-stack development
193 tasks, 16 categories, 918 prototype images, 1,255 test cases, all derived from real-world websites
Workflow-based agentic verification combining a GUI agent verifier with a VLM judge for reliable automated evaluation
Even state-of-the-art VLMs show substantial gaps, particularly on full-stack website development tasks