Paper Overview
Field: NLP Authors: Keya Hu, Linlu Qiu, Yiyang Lu Published: 2025-05-09 arXiv: 2505.07246
Summary
Diffusion and flow-based models have become the de facto approaches for generating continuous data, such as images and videos, and their success has attracted growing interest in applying them to language modeling. Unlike image-domain counterparts, today's leading diffusion language models (DLMs) primarily operate over discrete tokens.
This paper shows that continuous DLMs can be made effective with minimal adaptation to the discrete domain. The authors propose Embedded Language Flows (ELF), a class of diffusion models in continuous embedding space based on continuous-time Flow Matching. Key properties:
- Continuous embedding space: Unlike existing DLMs, ELF predominantly stays within the continuous embedding space until the final time step, where it maps to discrete tokens using a shared-weight network.
- Technique transfer: This formulation makes it straightforward to transfer well-established techniques from image-domain diffusion models, such as classifier-free guidance (CFG).
- Results: Experiments show that ELF significantly outperforms leading discrete and continuous DLMs, achieving better generation quality with fewer sampling steps.
Original Abstract (excerpt)
> Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them to language modeling. Unlike their image-domain counterparts, today's leading diffusion language models (DLMs) primarily operate over discrete tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded Language Flows (ELF), a class of diffusion models in continuous embedding space based on continuous-time Flow Matching...
*Auto-collected on 2026-05-13*