DaTMe: Dynamics-aware Token Merging for Efficient Diffusion Image Generation

Forum topic · 小凯 · 2026-10-17

Summary

DaTMe (Dynamics-aware Token Merging) is a training-free framework that accelerates diffusion models by deciding which tokens to merge based on underlying diffusion dynamics rather than prompt alignment. Existing token merging methods preserve prompt-aligned foreground regions, but this text-relevance criterion overlooks background objects and local details not explicitly mentioned in the prompt, degrading structural coherence. DaTMe instead builds a dynamics-aware importance map from Tweedie denoising estimates, combining a temporal signal (step-to-step variation of the predicted clean image, reflecting whether a token remains unresolved) and a structural signal (local high-frequency content such as edges and fine details). Experiments on PixArt-a and FLUX show that DaTMe maintains visual fidelity across the entire image while reducing redundant token computation, highlighting diffusion dynamics as an effective basis for token-level merging decisions in efficient image generation. Paper: arXiv 2610.22370.

Paper Overview

Field: Computer Vision (CV) Authors: SeungJu Cha, Ye-Chan Kim, HyunGee Kim, Sungho Koh, Dong-Jin Kim Published: 2026-10-15 arXiv: 2610.22370

Abstract (translated)

Token merging has emerged as an effective training-free strategy for accelerating diffusion models by reducing redundant token computation while minimizing generation degradation. A key challenge is determining which tokens can be safely merged and which should be preserved for subsequent computation. Existing methods address this by preserving prompt-aligned foreground regions, thereby protecting semantically relevant structures. However, this text-relevance criterion overlooks background objects and local details that are not explicitly mentioned in the prompt, leading to degraded structural coherence.

In this work, the authors propose Dynamics-aware Token Merging (DaTMe), a training-free framework that grounds token merging in the underlying diffusion dynamics. The key insight is that token importance should reflect whether a token remains unresolved during denoising and whether it carries structural information. DaTMe constructs a dynamics-aware importance map from Tweedie denoising estimates, combining:

Temporal signal: step-to-step variation of the predicted clean image, indicating tokens that are still unresolved in the denoising process.
Structural signal: local high-frequency content such as boundaries and fine details.

Experiments on PixArt-a and FLUX demonstrate that DaTMe preserves visual fidelity across the entire image, highlighting the role of diffusion dynamics in making effective token-level merging decisions for efficient generation.

Key Takeaways

Training-free acceleration for diffusion models via token merging.
Moves beyond prompt-relevance criteria to dynamics-based importance.
Combines temporal (denoising progress) and structural (high-frequency detail) signals.
Validated on PixArt-a and FLUX with maintained visual fidelity.

---

*Auto-collected on 2026-10-17*