[论文] Text-to-Image Generative Modeling Without Language Pretraining
研究领域: CV 作者: Keshigeyan Chandrasegaran, Yue Zhao, Armin W. Thomas, Ehsan Adeli, Jiajun Wu, Juan Carlos Niebles, Li Fei-Fei, Michael Poli 发布时间: 2026-10-19 arXiv…
论文概要
研究领域: CV 作者: Keshigeyan Chandrasegaran, Yue Zhao, Armin W. Thomas, Ehsan Adeli, Jiajun Wu, Juan Carlos Niebles, Li Fei-Fei, Michael Poli 发布时间: 2026-10-19 arXiv: 2610.26863
中文摘要
现代文生图（T2I）系统依赖互联网规模的语言预训练来进行文本条件图像生成。我们提出一个问题：强大的T2I生成是否必需语言预训练？我们引入了Halcyon——一系列在GPIC数据集（1亿图文对）上完全从零开始预训练的早期融合多模态自回归模型。Halcyon-0.52B以少18.2倍的预训练算力超过了LlamaGen-XL/8（48.0 vs. 63.3 gFD）。更进一步，Halcyon-1.6B以少275倍的预训练算力超过了JiT-T2I-XXL/8++（35.3 vs. 41.0 gFD）。对照实验表明，纯文本预训练并不能改善T2I生成质量。Halcyon还通过将字幕丢弃替换为I2T监督来学习图像字幕生成，在相同预训练预算下保持了有竞争力的T2I生成质量。我们证明了强大的T2I系统可以在没有互联网规模语言预训练的情况下从零训练。
原文摘要
Modern text-to-image (T2I) systems use internet-scale language pretraining for text-conditioned image generation. We ask whether language pretraining is necessary for strong T2I generation. We introduce Halcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scratch on GPIC, a dataset of 100M image-text pairs. Halcyon-0.52B outperforms LlamaGen-XL/8 (48.0 vs. 63.3 gFD) with 18.2\(\times\) less pretraining compute. Further, Halcyon-1.6B outperforms JiT-T2I-XXL/8++ (35.3 vs. 41.0 gFD) with 275\(\times\) less pretraining compute. Controlled experiments show that text-only pretraining does not improve T2I generation. Halcyon also learns image captioning by replacing caption dropout with I2T supervision, maintaining competitive T2I generation quality under the sa...
*自动采集于 2026-10-21*
#论文 #arXiv #CV #小凯