DC-Leap: Training-Free Acceleration of Diffusion LLMs via Draft-Guided Contiguous Leaping Decoding

Forum topic · 小凯 · 2026-09-02

Summary

DC-Leap is a training-free inference acceleration framework for diffusion large language models (dLLMs), developed by a team from Harbin Institute of Technology (Shenzhen) and Huawei Noah's Ark Lab, accepted at ICML 2026. Diffusion LLMs generate text by iteratively denoising a fully masked sequence, offering global context but suffering from many expensive iterative passes. DC-Leap addresses this with two mechanisms: Dynamic Contiguous Verification (DCV), which validates candidate tokens in a strict left-to-right causal chain so that moderately confident tokens can be accepted without introducing joint probability dependence errors, and Draft-Guided Decoding, which generates a fast draft of future tokens to serve as forward-looking context and semantic anchors. On MBPP long-sequence code generation, DC-Leap achieves a 53.19x speedup, and up to 105.02x when combined with KV-Cache optimization, while preserving task performance on models such as LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-7B-Instruct. Because it requires no weight modification or extra training, it is plug-and-play with existing dLLMs. Code is available at https://github.com/ffh-wyls/DC-Leap.

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

> Paper: DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding > Authors: Yanhua Jiao, Tianyi Wu, Xiaoxi Sun, Yulin Li, HuiLing Zhen, Libo Qin, Baotian Hu, Zhuotao Tian, Min Zhang (Harbin Institute of Technology, Shenzhen; Huawei Noah's Ark Lab) > arXiv: 2607.20467 | Venue: ICML 2026 > Code: https://github.com/ffh-wyls/DC-Leap

Key points

Background: Diffusion LLMs (dLLMs) generate text by starting from a fully masked sequence and progressively denoising it, enabling globally informed generation unlike the strictly left-to-right generation of autoregressive models (GPT-style). However, each denoising round requires a full forward pass over the entire sequence, making generation slow.
The problem with naive parallel decoding: Accepting many tokens per round introduces *JPDE* (Joint Probability Dependence Error) — predictions for later tokens may rely on earlier tokens that are not yet finalized. Existing methods counter this with high confidence thresholds, which severely limits per-round acceptance.
Mechanism 1 — Dynamic Contiguous Verification (DCV): Candidate tokens are verified in a strict left-to-right causal chain: each token is validated conditioned on the decoded prefix plus the previously verified tokens in the window. This cascade preserves causal consistency, so moderately confident tokens can be safely accepted without causing JPDE.
Mechanism 2 — Draft-Guided Decoding: A fast draft of future tokens is generated beyond the current window, providing forward-looking context. Draft tokens act as "semantic anchors"; even a partially accurate draft improves decoding decisions in the current window.
Training-free: DC-Leap works purely at inference time — no weight modifications, no extra training data — and is plug-and-play with existing dLLMs (validated on LLaDA and Dream).

Results

Models tested: LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-7B-Instruct
Benchmarks: MBPP (long-sequence code generation), GSM8K, HumanEval, and others
Speedups:
DC-Leap on MBPP (1024 tokens): 53.19x
DC-Leap + KV-Cache: 105.02x
Findings:

1. Gains are largest for long-sequence generation, where there is more room to "leap." 2. The acceleration is orthogonal to KV-Cache optimizations (e.g., dLLM-Cache): DC-Leap reduces the number of iterations, while KV-Cache reduces per-round computation. 3. Task performance remains comparable to baselines — the speedup does not trade away quality.

Why it matters

Diffusion LLMs offer theoretical advantages over autoregressive models — full bidirectional context, iterative refinement, and better controllability — but inference speed has been the main barrier to practical use. By delivering order-of-magnitude speedups at zero training cost, DC-Leap moves dLLMs significantly closer to production deployment.

References

1. Jiao, Y., Wu, T., Sun, X., et al. (2026). *DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding*. arXiv:2607.20467. ICML 2026. 2. Nie, S., et al. (2025). Large Language Diffusion Models. *ICLR*. 3. Kong, X., et al. (2025). LocalLeap: Accelerating dLLMs via Locality-Aware Parallel Decoding. 4. Ma, X., et al. (2025). dLLM-Cache: Efficient Key-Value Cache for Diffusion Language Models. 5. Chen, Y., et al. (2026). STDec: Spatio-Temporal Stability Guided Decoding for dLLMs. arXiv:2604.06330.