From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Image Generation

Forum topic · 小凯 · 2026-08-20

Summary

A new paper on arXiv (2608.18076) proposes a capability-driven data infrastructure for large-scale image generation. Unlike conventional pipelines that optimize task-specific datasets in isolation, the approach couples capability-specific supervision construction with capability-aligned curriculum scheduling. Three specialized, interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, while caption experts align text-to-image (T2I) and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task composition, visual concept distribution, data quality, and image resolution along the dependency order of capability acquisition, closed-looped by capability-aware evaluation via targeted retrieval, expert curation, and gap-aware resampling. The framework curates 440 million T2I images, 120 million editing pairs, and over 27 million image-entity pairs. Using this infrastructure, the authors train 3B- and 6B-parameter multimodal diffusion models from scratch, demonstrating broad visual coverage, diverse rendering styles, and effective transfer across generative capabilities.

Paper Overview

Field: Computer Vision arXiv: 2608.18076

Abstract (Translated)

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

The authors present a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Its three specialized yet interoperable data engines build complementary relational supervision for:

Text-image grounding
Inter-image transformation
Image-knowledge association

Caption experts align text-to-image and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task composition, visual concept distribution, data quality, and image resolution along the dependency order of capability acquisition. Capability-aware evaluation closes the loop through targeted retrieval, expert curation, and gap-aware resampling.

Data Scale

440 million text-to-image images
120 million editing pairs
27+ million image-entity pairs

Results

Using this infrastructure, multimodal diffusion models with 3 billion and 6 billion parameters were trained from scratch. Experiments show broad visual coverage, diverse rendering styles, and effective transfer across generative capabilities.

---

*Auto-collected on 2026-08-20.*