[论文] Representation Conditioning for Generation

论文概要

研究领域: CV
作者: Saeed Razavi, Xingjian Leng, Liang Zheng
发布时间: 2026-10-14
arXiv: 2610.20608

中文摘要

表示对齐方法能有效加速扩散Transformer的训练。以REPA为例，它使用预训练视觉编码器通过MLP投影头监督中间层，训练后丢弃投影头，旨在将视觉编码器的语义引入扩散骨干网络。虽然通过反向传播对齐损失可以在早期层实现这一目标，但仅受速度预测损失监督的后续层能否很好地获取这些语义并无保证。我们提出一个简单的解决方案——表示条件化（RECON），以改善后续层的语义。训练时将预训练视觉编码器的token输入到后续层；推理时由于没有干净图像，我们复用MLP头（作为视觉编码器的代理）并将其输出token投影到后续层。我们的方法不增加任何参数，且收敛速度快得多：RECON比SiT快63倍，比REPA快25倍。

原文摘要

Representation-alignment methods effectively accelerate the diffusion transformer training. REPA, for example, uses a pretrained vision encoder to supervise an intermediate layer through an MLP projection head which is discarded after training. It aims to import semantics from the vision encoder to the diffusion backbone. While this objective is achieved at early layers by back-propagating the alignment loss, there is no guarantee that later layers, supervised by the velocity prediction loss alone, can well access such semantics. We propose a simple solution, representation conditioning (RECON), to improve semantics in later layers. During training, we feed the pretrained vision encoder tokens into later layers; during inference, because we no longer have the clean image, we reuse the MLP ...

自动采集于 2026-10-16

#论文 #arXiv #CV #小凯