From Corpora to Co-Evolving Capabilities: A Capability-Centric Data Infrastructure for Large-Scale Image Generation

Forum topic · 小凯 · 2026-08-20

Summary

This paper introduces a capability-driven data infrastructure for large-scale image generation that shifts focus from task-specific dataset curation to organizing heterogeneous supervision according to dependencies among generative capabilities. The framework couples capability-specific supervision construction with capability-aligned curriculum scheduling. Three specialized, interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, while caption experts align text-to-image and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task composition, visual concept distribution, data quality, and image resolution along capability acquisition dependencies, and capability-aware evaluation closes the loop via targeted retrieval, expert construction, and gap-aware resampling. The pipeline curated 440 million text-to-image images, 120 million editing pairs, and over 27 million image-entity pairs. Using this infrastructure, the authors trained 3B and 6B parameter multimodal diffusion models from scratch, demonstrating broad visual coverage, diverse rendering effects, and effective transfer across generation capabilities. arXiv: 2608.18076.

论文概要 (Paper Overview)

Field: Computer Vision arXiv: 2608.18076

Abstract

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

This work presents a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling.

Key Components

Three specialized, interoperable data engines build complementary relational supervision for:
Text-image grounding
Inter-image transformation
Image-knowledge association
Caption experts align text-to-image (T2I) and editing supervision across tasks and granularities.
Multi-stage curriculum co-evolves task composition, visual concept distribution, data quality, and image resolution along the dependency order of capability acquisition.
Capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling.

Scale and Results

The framework curated:

440 million text-to-image images
120 million editing pairs
Over 27 million image-entity pairs

Using this infrastructure, the authors trained 3B and 6B parameter multimodal diffusion models from scratch. Experiments demonstrate broad visual coverage, diverse rendering effects, and effective transfer across generation capabilities.

---

*Originally posted on zhichai.net, auto-collected 2026-08-20.*