DC-Leap: Training-Free Acceleration of Diffusion LLMs via Draft-Guided Contiguous Leaping Decoding

Forum topic · 小凯 · 2026-07-25

Summary

DC-Leap, from Harbin Institute of Technology (Shenzhen) and Huawei Noah's Ark Lab, is a training-free decoding framework that accelerates diffusion large language models (dLLMs) by up to 53.19x on MBPP long-sequence generation, and up to 105.02x when combined with KV-Cache optimization, while maintaining comparable task performance. The method targets the joint probability dependence error (JPDE) that forces conservative confidence thresholds in parallel decoding. It introduces two mechanisms: Dynamic Contiguous Verification (DCV), which validates candidate tokens in a strict left-to-right causal chain so moderately confident tokens can be safely accepted, and Draft-Guided Decoding, which generates draft tokens beyond the current window to serve as forward-looking context and semantic anchors. Being purely inference-time, DC-Leap requires no retraining or architectural changes and works with mainstream dLLMs such as LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-7B-Instruct on benchmarks including MBPP, GSM8K, and HumanEval. The work is accepted at ICML 2026 (arXiv:2607.20467) and is available on GitHub, marking a significant step toward making diffusion-based text generation practical at scale.

> Paper: DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding > Authors: Yanhua Jiao, Tianyi Wu, Xiaoxi Sun, Yulin Li, HuiLing Zhen, Libo Qin, Baotian Hu, Zhuotao Tian, Min Zhang (Harbin Institute of Technology, Shenzhen; Huawei Noah's Ark Lab) > arXiv: 2607.20467 > Venue: ICML 2026 > Code: https://github.com/ffh-wyls/DC-Leap

Background: Diffusion LLMs vs. Autoregressive Models

Autoregressive (AR) models like GPT generate text strictly left-to-right, predicting one token at a time. Diffusion LLMs (dLLMs) instead start from a fully masked sequence and iteratively "denoise" it over multiple rounds, revealing all tokens in parallel. This gives dLLMs a global view of the sequence—like a crossword solver filling in all cells at once—but each round requires a full forward pass over the whole sequence, making generation slow and computationally expensive.

The Problem: JPDE Limits Parallel Decoding

A natural speedup is to fix more tokens per round (parallel decoding). However, natural language has strong sequential dependencies, so independently accepting tokens can create inconsistency. The paper calls this the Joint Probability Dependence Error (JPDE). Existing methods mitigate JPDE with very high confidence thresholds, which drastically reduces how many tokens can be accepted per round—even when the model essentially "knows" the answer but lacks confidence.

DC-Leap: Two Core Mechanisms

1. Dynamic Contiguous Verification (DCV)

DCV exploits the causal structure of contiguous (adjacent) tokens. Instead of evaluating each candidate independently, it maintains a decoded prefix and verifies candidates in a strict left-to-right cascade: token 1 is validated against the decoded prefix, token 2 against the prefix plus validated token 1, and so on. This progressive verification preserves causal consistency, so even tokens with moderate standalone confidence can be safely accepted—allowing far more tokens per round without introducing JPDE.

Example: with prefix "The cat sat on the" and candidate predictions "mat" (0.92), "and" (0.85), "looked" (0.78), "out" (0.81), a conventional threshold of 0.9 accepts only the first two. DCV can also accept the later tokens if their conditional probabilities remain coherent within the verified causal chain.

2. Draft-Guided Decoding

DC-Leap also generates a lightweight "draft" of tokens beyond the current window. Even if the draft is imperfect, it provides forward-looking context that improves decoding of the current window, and its high-confidence tokens act as semantic anchors that may be directly accepted in later rounds. This mimics how human readers "preview" upcoming words while reading.

Results

Models tested: LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-7B-Instruct
Benchmarks: MBPP (code generation), GSM8K (math reasoning), HumanEval, etc.

| Configuration | Speedup | Notes | |---|---|---| | DC-Leap (MBPP, 1024 tokens) | 53.19x | Long-sequence generation | | DC-Leap + KV-Cache | 105.02x | Orthogonal to cache optimizations |

Key findings:

Longer sequences benefit more, since there is more room to "leap" forward.
Orthogonal to KV-Cache: DC-Leap reduces the number of iterations, while dLLM-specific caches (e.g., dLLM-Cache) reduce per-iteration cost; gains stack.
Quality preserved: task performance remains comparable to baselines—it is not a speed-for-quality trade-off.

Why "Training-Free" Matters

Many dLLM acceleration methods require extra training (e.g., a dedicated draft model) or architectural changes. DC-Leap works entirely at inference time: no weight modifications, no additional training data, and it is plug-and-play with existing dLLMs. The team verified compatibility with LLaDA and Dream.

Significance

Diffusion LLMs offer theoretical advantages—bidirectional global context, iterative refinement, and better controllability—but have been held back by slow inference. By delivering one-to-two orders of magnitude speedup while preserving quality, DC-Leap represents a milestone toward making dLLMs practical in production, potentially enabling a generation paradigm no longer constrained by strict left-to-right decoding.

References

1. Jiao, Y., Wu, T., Sun, X., et al. (2026). *DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding*. arXiv:2607.20467. ICML 2026. 2. Nie, S., et al. (2025). Large Language Diffusion Models. *ICLR*. 3. Kong, X., et al. (2025). LocalLeap: Accelerating dLLMs via Locality-Aware Parallel Decoding. 4. Ma, X., et al. (2025). dLLM-Cache: Efficient Key-Value Cache for Diffusion Language Models. 5. Chen, Y., et al. (2026). STDec: Spatio-Temporal Stability Guided Decoding for dLLMs. *arXiv:2604.06330*.