DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

Forum topic · 小凯 · 2026-04-30

Summary

DV-World is a benchmark of 260 tasks designed to evaluate data visualization (DV) agents across real-world professional lifecycles, addressing limitations of existing benchmarks such as code-sandbox confinement, single-language creation-only tasks, and the assumption of perfectly specified intent. The benchmark spans three domains: DV-Sheet for native spreadsheet manipulation including chart and dashboard creation and diagnostic repair; DV-Evolution for adapting and restructuring reference visual artifacts across diverse programming paradigms; and DV-Interact for proactive intent alignment using a user simulator that mimics ambiguous real-world requirements. Evaluation combines a hybrid framework integrating tabular numerical alignment for numerical precision with MLLM-as-a-Judge for semantic-visual assessment against rubrics. Experiments show that state-of-the-art models score below 50% overall, revealing critical deficiencies in handling the complexities of real-world data visualization. Paper by Jinxiang Meng, Shaoping Huang, Fangyu Lei, et al., available at arXiv:2504.21252.

Paper Overview

Research Area: NLP Authors: Jinxiang Meng, Shaoping Huang, Fangyu Lei, et al. arXiv: 2504.21252

Abstract

Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benchmarks often suffer from code-sandbox confinement, single-language creation-only tasks, and assumption of perfect intent. To bridge these gaps, the authors introduce DV-World, a benchmark of 260 tasks designed to evaluate DV agents across real-world professional lifecycles.

DV-World spans three domains:

DV-Sheet: native spreadsheet manipulation, including chart and dashboard creation as well as diagnostic repair.
DV-Evolution: adapting and restructuring reference visual artifacts to fit new data across diverse programming paradigms.
DV-Interact: proactive intent alignment with a user simulator that mimics real-world ambiguous requirements.

Evaluation Framework

A hybrid evaluation framework integrates:

Tabular numerical alignment to guarantee numerical precision.
MLLM-as-a-Judge for semantic-visual assessment according to scoring rubrics.

Findings

Experiments show that state-of-the-art models achieve less than 50% overall performance, exposing critical deficiencies in handling the complex challenges of real-world data visualization.

*Paper link*: https://arxiv.org/abs/2504.21252