DC-Leap: Training-Free Acceleration of Diffusion LLMs via Draft-Guided Contiguous Leaping Decoding
> Paper: DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding > Authors: Yanhua Jiao, Tianyi Wu, Xiaoxi Sun, Yulin Li, HuiLing Zhen, Libo Qin, Baotian Hu, Zhuotao Tian, Min Zhang (Harbin Institute of Technology, Shenzhen; Huawei Noah's Ark Lab) > arXiv: 2607.20467 > Venue: ICML 2026 > Code: https://github.com/ffh-wyls/DC-Leap
Background: Autoregressive vs. Diffusion Language Models
Autoregressive (AR) models such as GPT generate text strictly left-to-right, predicting one token at a time based on preceding context. Diffusion large language models (dLLMs) instead start from a fully masked sequence and iteratively "denoise" it: each round the model predicts all masked positions, fixes the most confident predictions, and repeats until the sequence is complete. This enables true global, bidirectional context usage, but requires many expensive full-sequence forward passes, making generation slow.
The Problem: JPDE in Parallel Decoding
The obvious way to speed up dLLMs is parallel decoding — fixing more tokens per iteration. However, this introduces the Joint Probability Dependence Error (JPDE): when multiple tokens are accepted simultaneously, each prediction may have been conditioned on neighboring tokens that are not yet finalized, violating the sequential dependencies of natural language. Existing methods cope by setting very high confidence thresholds, which is conservative and limits the tokens accepted per round.
DC-Leap's Two Mechanisms
1. Dynamic Contiguous Verification (DCV)
DCV exploits the causal structure of contiguous token windows. It maintains a decoded prefix and validates candidate tokens in strict left-to-right order:
1. Token 1 is verified against the decoded prefix 2. Token 2 is verified against the prefix + verified token 1 3. Token 3 against prefix + tokens 1–2, and so on
This cascaded verification preserves causal consistency, so moderately confident tokens (e.g., 0.78 confidence) can be safely accepted as long as they are self-consistent within the causal chain — greatly increasing tokens accepted per round without introducing JPDE.
2. Draft-Guided Decoding
DC-Leap generates a quick "draft" of tokens beyond the current decoding window. Even an imperfect draft provides forward-looking context that improves decisions in the current window, leveraging dLLMs' bidirectional attention. High-confidence draft tokens act as semantic anchors and may be directly accepted in later rounds, further reducing iterations.
Key Results
- Models tested: LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-7B-Instruct
- Benchmarks: MBPP (code generation), GSM8K (math reasoning), HumanEval
- Gains grow with sequence length: longer outputs offer more room for leaping.
- Orthogonal to KV-Cache: DC-Leap reduces iteration count while KV-Cache reduces per-round computation; the two compose.
- Quality preserved: task scores remain comparable to baseline decoding.
| Configuration | Speedup | Notes | |---|---|---| | DC-Leap (MBPP, 1024 tokens) | 53.19× | Long-sequence generation | | DC-Leap + KV-Cache | 105.02× | Orthogonally stackable with cache optimizations |
Key findings:
Why "Training-Free" Matters
DC-Leap requires no weight modification, no extra training data, and no auxiliary draft model — it is a pure inference-stage strategy, plug-and-play with any existing dLLM. This contrasts with acceleration methods that require training dedicated draft models or architectural changes.
Significance
By accelerating dLLM inference by one to two orders of magnitude, DC-Leap moves diffusion LLMs from an interesting-but-slow research direction toward practical deployment, potentially unlocking AR-free generation paradigms with global context, iterative refinement, and better controllability.
References
1. Jiao, Y., Wu, T., Sun, X., et al. (2026). *DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding*. arXiv:2607.20467. ICML 2026. 2. Nie, S., et al. (2025). Large Language Diffusion Models. *ICLR*. 3. Kong, X., et al. (2025). LocalLeap: Accelerating dLLMs via Locality-Aware Parallel Decoding. 4. Ma, X., et al. (2025). dLLM-Cache: Efficient Key-Value Cache for Diffusion Language Models. 5. Chen, Y., et al. (2026). STDec: Spatio-Temporal Stability Guided Decoding for dLLMs. arXiv:2604.06330.