[论文] Know, then Draw: A New Learning Paradigm for Visual Generation

论文概要

研究领域: CV / 视觉生成
作者: Yucheng Mao, Chongjian Ge, Zhuowen Tu, Hao Tan, Hanwen Jiang
发布时间: 2026-10-27
arXiv: 2610.37626

中文摘要

视觉生成同时需要高层语义知识和细粒度像素细节。本文提出一种学习范式：将知识获取分配给自回归预训练，将高保真渲染分配给扩散后训练。在此框架下，团队研究了知识获取和渲染的目标、表示和扩展规律。受控对比实验确定自回归是更强的下游渲染预训练目标，同时改善学习效率和最终生成质量。团队还发现AR-only生成质量与最终性能不一定相关，语义质量更重要。这促使他们训练一种纯语义tokenizer——仅用语言监督，无需图像解码器或重建损失。团队建立了知识获取和渲染的计算最优scaling law，发现自回归预训练降低了拟合的渐近渲染损失。在这些规律的指导下，仅用6,600 H100 GPU小时训练了一个3B参数的文本到图像模型，在GenEval上达到86分，在DPG-Bench上达到87分。

原文摘要

Visual generation requires both high-level semantic knowledge and fine-grained pixel details. We propose a learning paradigm that assigns these roles to autoregressive pretraining for knowledge acquisition and diffusion post-training for high-fidelity rendering. Within this framework, we study the objective, representation, and scaling of knowledge acquisition and rendering. Controlled comparisons identify autoregression as the stronger pretraining objective for downstream rendering, improving both learning efficiency and final generation quality. We further find that AR-only generation quality does not necessarily correlate with final performance; semantic quality matters more. This motivates a purely semantic tokenizer trained with language supervision alone, without an image decoder or reconstruction loss. We establish compute-optimal scaling laws for knowledge acquisition and rendering, and find that autoregressive pretraining lowers the fitted asymptotic rendering loss compared with pixel diffusion models trained from scratch. Guided by the laws, we train a 3B-parameter text-to-image model with only 6,600 H100 GPU-hours, achieving 86 on GenEval and 87 on DPG-Bench and validating the proposed paradigm at scale.

自动采集于 2026-10-29

#论文 #arXiv #AI #小凯