DFlash, Diffusion Language Models (dLLM), MTP and Speculative Decoding: A Deep Research Report

Forum topic · ✨步子哥 · 2026-06-12

Summary

This in-depth technical report from zhichai.net analyzes three converging approaches to breaking the serial bottleneck of autoregressive LLM decoding: diffusion language models (dLLMs), block-diffusion speculative decoding (DFlash), and multi-token prediction (MTP). The report traces the dLLM lineage from D3PM and MDLM through LLaDA 8B, Dream 7B and LLaDA 2.0 (100B), then examines DFlash, which repositions a diffusion model as a lightweight draft generator for speculative decoding. DFlash's two core innovations—block diffusion parallel drafting (block_size=16) and per-layer KV injection of target-model hidden features—deliver lossless speedups of 4.86x on Qwen3-8B greedy decoding and up to 6.17x on MATH-500, roughly 2.5x faster than EAGLE-3. The report also covers FastMTP's train-inference alignment (82% improvement over vanilla MTP), production benchmarks on SGLang with B200 GPUs, performance degradation at high concurrency, and open problems including draft-model training cost, MTP-diffusion co-training, adaptive block sizes, and multimodal extensions. Key conclusion: diffusion models need not replace autoregressive models—as smart drafters they already change inference architecture design.

DFlash, Diffusion Language Models (dLLM), MTP and Speculative Decoding — Deep Research Report

*Deep Research Report | Scope: dLLM architecture evolution, DFlash block-diffusion speculative decoding, multi-token prediction (MTP), and LLM inference acceleration. Originally published on zhichai.net.*

Key points

The bottleneck: Autoregressive decoding generates one token per forward pass. For ~7B models, single-token GPU utilization is typically under 5% — a fundamental memory-wall/throughput bottleneck (Xia et al., ACL 2024).
Three routes forward: (1) speculative decoding (draft + parallel verify), (2) diffusion language models (parallel denoising instead of token-by-token generation), (3) multi-token prediction (MTP) as a modified training objective.
DFlash's paradigm shift: a diffusion model does not need to compete with AR models on generation quality — it only needs to be a good *drafter*, turning diffusion from an "AR replacement" into an "AR accelerator."
DFlash results: average 4.86x speedup on Qwen3-8B (greedy), up to 6.17x on MATH-500 vs EAGLE-3's 2.30x; acceptance length τ ≈ 7.84 on MATH-500 vs 2.65–3.30 for EAGLE-3(16). At temperature=1: 4.03x average. Production (SGLang + B200): 5.1x at concurrency 1, degrading to 2.8x at 32 concurrent requests (GPU saturation, common to all speculative methods).
dLLM maturity: from D3PM (2021) → MDLM → DiffuLLaMA → LLaDA 8B → Dream 7B → LLaDA 2.0 (100B). Dream 7B (AR-weight initialization + context-adaptive noise rescheduling) surpasses same-scale AR models on planning tasks like Countdown and Sudoku.
MTP's two roles: training-side (better long-range modeling, +17% on code generation per Gloeckle et al. 2024) and inference-side (draft heads for speculative decoding, e.g., Gemma 4). FastMTP's train-inference alignment + dynamic vocabulary compression improves MTP speedup by 82% (2.03x average, lossless).

1. Background

Speculative decoding

A lightweight draft model proposes k candidate tokens; the target model verifies them in a single forward pass; accepted tokens are kept, mismatches are discarded, and a bonus token is generated. If τ tokens are accepted per round, speedup is approximately:

\eta = \frac{T_{\text{target}}}{\tau} \cdot (T_{\text{draft}} + T_{\text{verify}})

Method families (Xia et al., 2024): independent draft models (SpecDec, SpecInfer, ~2x), self-speculative (Medusa, Lookahead, ~2–3x), feature-level drafting (EAGLE series, ~2–3x), and diffusion drafting (DFlash, SpecDiff, ~5–6x).

Diffusion language models

Discrete diffusion (D3PM, Austin et al. 2021) defines forward noising and reverse denoising over discrete token states. Training: randomly mask tokens at sampled timesteps; inference: start from all-[MASK] and iteratively denoise. Key models:

| Time | Model | Scale | Contribution | |---|---|---|---| | 2021 | D3PM | ~100M | Discrete diffusion foundations (NeurIPS 2021) | | 2024.06 | MDLM | ~300M | Simple masked diffusion LM (NeurIPS 2024) | | 2024.10 | DiffuLLaMA | 7B | Adapting AR weights to diffusion (ICLR 2025) | | 2025.02 | LLaDA | 8B | First large-scale diffusion LLM | | 2025.04 | Dream 7B | 7B | Planning tasks exceed AR; strongest open dLLM | | 2025.12 | LLaDA 2.0 | 100B | Diffusion scaled to 100B | | 2026.02 | DFlash | 5-layer draft | Diffusion + speculative decoding fusion |

Dream 7B (HKU NLP + Huawei Noah's Ark) initializes from Qwen2.5-7B weights and applies context-adaptive noise rescheduling; on planning tasks it sometimes outperforms even DeepSeek V3 (671B).

Inference acceleration for standalone dLLMs: FlashDLM (~12x via KV-cache reuse + guided diffusion, training-free), Fast-dLLM (~10x), consistency distillation (CD4LM, IMDM). Note: these speedups target dLLM standalone inference and are not directly comparable to DFlash's lossless speculative speedup.

2. DFlash: block-diffusion speculative decoding

Architecture

Prefill: target model processes the prompt; hidden features from 5 uniformly-spaced layers are concatenated and projected via $W_c \in \mathbb{R}^{D \times 5D}$ , then RMSNorm-normalized into a target context feature.
Draft + verify loop: the target context feature is injected into the draft model's KV cache at every layer; block diffusion generates 16 candidate tokens in one forward pass; the target model verifies all candidates at once.

KV injection (vs EAGLE-3)

EAGLE-3 injects target features only at the first layer's input; DFlash injects them into the Key/Value projections of every draft layer:

Q_i = W_i^Q \cdot H_d

K_i = [W_i^K \cdot H_t \;|\; W_i^K \cdot H_d]

V_i = [W_i^V \cdot H_t \;|\; W_i^V \cdot H_d]

Injected features persist across drafting iterations and enable acceptance length to scale with draft depth — something EAGLE-3 cannot achieve (signal dilutes with depth). Ablations show KV injection beats input-fusion in all settings.

Block diffusion drafting

A 5–8 layer draft model generates the whole block (default block_size=16) with a single denoising step; drafting latency is independent of block length, unlike AR drafting ( $\propto \gamma$ ). Bidirectional (full) attention within the block exploits two-sided context.

Training innovations

1. Random anchor sampling: anchor positions are sampled randomly in the response rather than uniform blocks, matching inference-time conditioning on clean target tokens (+13–18% speedup in ablations). 2. Exponentially decaying loss weights: $w_k = \exp(-(k-1)/\gamma)$ with $\gamma=7$ for block_size=16, since early-position errors invalidate all later tokens. 3. Block-size generalization: models trained with block_size=16 generalize to smaller inference blocks (8); the reverse does not hold.

Benchmarks (greedy, temperature=0, Qwen3-8B)

| Benchmark | Base | EAGLE-3(16) | EAGLE-3(60) | DFlash(16) | |---|---|---|---|---| | GSM8K | 1x | 2.13x | 2.26x | 5.20x | | MATH-500 | 1x | 2.18x | 2.30x | 6.17x | | HumanEval | 1x | 2.48x | 2.39x | 5.20x | | MT-Bench | 1x | 1.94x | 1.90x | 2.79x |

Long-context: after fine-tuning on LongAlign-10K, 16K-context performance exceeds 4K (Qasper: 3.57x → 6.00x).

Production (SGLang + B200): Qwen3-8B/Math500 — 5.1x (1 concurrency), 4.5x (8), 2.8x (32); Qwen3-Coder-30B/HumanEval — 3.5x / 3.2x / 3.1x.

Ecosystem

Pre-trained draft models available for Gemma-4-31B, Qwen3.5-27B/122B, Qwen3.6-27B, Kimi-K2.6, GPT-OSS-120B, LLaMA-3.1-8B (with DeepSeek-V4-Flash and GLM-5.1 announced). Integrated into vLLM (v0.20.1+), SGLang, Transformers, and MLX (Apple Silicon).

3. Why DFlash beats EAGLE-3

1. Decoupled drafting latency: 5-layer DFlash drafting 16 tokens is faster than 1-layer EAGLE-3 drafting 8 tokens. 2. Per-layer KV injection maintains target conditioning at all depths. 3. Bidirectional block context improves draft quality over causal attention.

4. MTP and its complementarity with diffusion drafting

MTP training-side: +17% on code generation, 3x inference potential (Meta, Gloeckle et al. 2024); used in DeepSeek-V3's training objective.
MTP inference-side: draft heads for speculation (Gemma 4). FastMTP fixes the train/inference distribution mismatch via self-distillation, position-shared weights, and language-aware dynamic vocabulary compression → 2.03x average, +82% over vanilla MTP.
Complementarity: MTP drafting is recursive/autoregressive (~2–3x ceiling, built-in deployment); DFlash drafting is parallel with bidirectional context (~5–6x ceiling, requires a separate draft model). Combining MTP objectives with diffusion drafting remains unexplored.

5. Open problems and future directions

1. Speedup decay under high concurrency (2.8x at 32 requests) — GPU saturation, inherent to all speculative methods. 2. Per-target draft training cost — DFlash training is lightweight (6 epochs @ 800K samples) but each target model needs its own drafter; zero-shot/transferable drafters are an open direction. 3. Deeper AR-diffusion fusion — natively embedding diffusion capability into AR architectures for end-to-end unified training. 4. Adaptive block sizes — large blocks for easy contexts, small for hard reasoning. 5. MTP-diffusion co-training, diffusion chain-of-thought (building on Dream 7B's planning strengths), multimodal diffusion speculation, and edge deployment via MLX.

6. Conclusion

Diffusion LLMs have matured (LLaDA 2.0 at 100B), DFlash sets a new speculative-decoding SOTA (5–6x lossless, ~2.5x over EAGLE-3), and MTP is a valuable complementary technique. The field is shifting from "replacing AR" to "augmenting AR" — diffusion + speculation + MTP together represent the most promising current direction for LLM inference acceleration, a shift being rapidly adopted by mainstream frameworks like vLLM and SGLang.

*Note: figures above are as reported in the original post; the source article was truncated at the conclusion section.*