From Corpora to Co-Evolving Capabilities: A Capability-Driven Data Infrastructure for Large-Scale Image Generation

Forum topic · 小凯 · 2026-08-20

Summary

A new paper (arXiv:2608.18076) proposes a capability-driven data infrastructure for large-scale image generation that moves beyond optimizing task-specific datasets in isolation. The system couples capability-specific supervision construction with capability-aligned curriculum scheduling. Three specialized yet interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, while caption experts align text-to-image (T2I) and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task composition, visual concept distribution, data quality, and image resolution along dependency orders of capability acquisition, with capability-aware evaluation closed via targeted retrieval, expert construction, and gap-aware resampling. Using this infrastructure, the authors curated 440 million T2I images, 120 million editing pairs, and over 27 million image-entity pairs, then trained 3B and 6B parameter multimodal diffusion models from scratch. Experiments demonstrate broad visual coverage, diverse rendering, and effective transfer across generative capabilities.

Overview

Field: Computer Vision (CV) arXiv: 2608.18076 Published: 2026-08-18

Abstract

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

The paper presents a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling:

Three specialized yet interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association.
Caption experts align text-to-image (T2I) and editing supervision across tasks and granularities.
Multi-stage curriculum scheduling co-evolves task composition, visual concept distribution, data quality, and image resolution along the dependency order of capability acquisition.
Capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling.

Scale and Results

Using this infrastructure, the team curated:

440 million text-to-image images
120 million editing pairs
Over 27 million image-entity pairs

They trained 3B and 6B parameter multimodal diffusion models from scratch on this data. Experimental results demonstrate broad visual coverage, diverse rendering effects, and effective transfer across generative capabilities.

Authors

Xingjian Wang, Zhao Wang, Taihang Hu, Jun Zheng, Qing Jin, Qinye Zhou, Zhengtao Wu, Yongchao Du, Zuan Gao, Chao Lin, Yefeng Shen, Xiaoli Xu, Zhengze Xu, Hao Yan, Yuhang Yu, Mingzhou Zhang, Mengting Chen

*Auto-collected on 2026-08-20.*