[论文] Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generati...

论文概要

研究领域: CV 作者: SeungJu Cha, Ye-Chan Kim, HyunGee Kim, Sungho Koh, Dong-Jin Kim 发布时间: 2026-10-15 arXiv: 2610.22370

中文摘要

Token 合并已成为一种有效的免训练策略，通过减少冗余 token 计算来加速扩散模型。关键挑战是确定哪些 token 可以安全合并。现有方法通过保留与提示对齐的前景区域来保护语义相关结构，但这种文本相关性标准忽略了背景物体和提示中未明确提及的局部细节，导致结构连贯性下降。我们提出动态感知 Token 合并（DaTMe），这是一个将 token 合并建立在底层扩散动力学基础上的免训练框架。核心洞察是：token 的重要性应反映其在去噪过程中是否仍未解决，以及是否携带结构信息。DaTMe 从 Tweedie 去噪估计构建动态感知重要性图，结合时间信号（预测干净图像的步间变化）和结构信号（局部高频内容如边界和细节）。在 PixArt-a 和 FLUX 上的实验表明，DaTMe 在整个图像上保持视觉保真度，突出了扩散动力学作为高效生成中有效 token 级合并决策的作用。

原文摘要

Token merging has emerged as an effective training-free strategy for accelerating diffusion models by reducing redundant token computation while minimizing generation degradation. A key challenge is determining which tokens can be safely merged and which should be preserved for subsequent computation. Existing method addresses this by preserving prompt-aligned foreground regions, thereby protecting semantically relevant structures. However, this text-relevance criterion overlooks background objects, local details that are not explicitly mentioned in the prompt, leading to degraded structural coherence. In this work, we propose Dynamics-aware Token Merging (DaTMe), a training-free framework that grounds token merging in the underlying diffusion dynamics. Our key insight is that token import...

#论文 #arXiv #CV #小凯