From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Image Generation

Forum topic · 小凯 · 2026-08-20

Summary

A forum post shares the arXiv paper 'From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Image Generation' (arXiv:2608.18076, computer vision). The paper argues that conventional image-generation data pipelines optimize task-specific datasets in isolation, and proposes a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Three interoperable data engines build relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, with caption experts aligning text-to-image and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task mixtures, visual concept distributions, data quality, and image resolution, closed-looped via capability-aware evaluation. The resulting dataset spans 440 million text-to-image images, 120 million editing pairs, and over 27 million image-entity pairs, used to train 3B and 6B multimodal diffusion models from scratch, demonstrating broad visual coverage, diverse rendering, and effective transfer across generation capabilities.

Paper Overview

Field: Computer Vision (CV) arXiv: 2608.18076

Abstract

Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities.

The paper presents a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Its three specialized yet interoperable data engines build complementary relational supervision for:

Text-image grounding
Inter-image transformation
Image-knowledge association

Caption experts align T2I and editing supervision across tasks and granularities. A multi-stage curriculum co-evolves task mixtures, visual concept distributions, data quality, and image resolution along the dependency order of capability acquisition, while capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling.

Scale and Results

Using this infrastructure, the authors curated:

440 million text-to-image images
120 million editing pairs
Over 27 million image-entity pairs

They trained 3B and 6B parameter multimodal diffusion models from scratch. Experiments demonstrate broad visual coverage, diverse rendering effects, and effective transfer across generative capabilities.

---

*Auto-collected on 2026-08-20.*