From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Image Generation

Forum topic · 小凯 · 2026-08-20

Summary

A new paper on arXiv (2608.18076) introduces a capability-driven data infrastructure for large-scale image generation. Instead of optimizing task-specific datasets in isolation, the approach organizes heterogeneous supervision according to dependencies among generative capabilities. The framework combines capability-specific supervision construction with capability-aligned curriculum scheduling, using three specialized and interoperable data engines that build relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, with caption experts aligning text-to-image (T2I) and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task mixtures, visual concept distributions, data quality, and image resolution along the dependency order of capability acquisition, while capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling. The infrastructure curated 440 million T2I images, 120 million editing pairs, and over 27 million image-entity pairs. Using it, the authors trained 3B and 6B parameter multimodal diffusion models from scratch, demonstrating broad visual coverage, diverse rendering styles, and effective transfer across generative capabilities.

论文概要

Research area: Computer Vision (CV) Authors: Xingjian Wang, Zhao Wang, Taihang Hu, Jun Zheng, Qing Jin, Qinye Zhou, Zhengtao Wu, Yongchao Du, Zuan Gao, Chao Lin, Yefeng Shen, Xiaoli Xu, Zhengze Xu, Hao Yan, Yuhang Yu, Mingzhou Zhang, Mengting Chen Published: 2026-08-18 arXiv: 2608.18076

Summary

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

The paper presents a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Its key components:

Three specialized yet interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association.
Caption experts align T2I and editing supervision across tasks and granularities.
Multi-stage curriculum co-evolves task mixtures, visual concept distributions, data quality, and image resolution along the dependency order of capability acquisition.
Capability-aware evaluation closes the loop via targeted retrieval, expert construction, and gap-aware resampling.

Scale

The framework curated:

440 million text-to-image (T2I) images
120 million editing data pairs
Over 27 million image-entity pairs

Results

Using this infrastructure, the authors trained 3B and 6B parameter multimodal diffusion models from scratch. Experiments demonstrate broad visual coverage, diverse rendering effects, and effective transfer across generative capabilities.

---

Original abstract (excerpt):

> Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities...

Paper link: https://arxiv.org/abs/2608.18076

*Auto-collected on 2026-08-20.*

From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Image Generation

Summary

论文概要

Summary

Scale

Results

Tags