Key points
- Problem addressed: Speculative decoding for Block Diffusion LLMs is bottlenecked by a draft model that (a) shares one fused target-context feature across all draft layers, and (b) saturates in acceptance length when depth is increased. These two limits are coupled, so scaling draft capacity stalls.
- Core idea: "Flare out" DFlash's narrow bottleneck by giving every draft layer its own view of the target model, separating the attention subspaces used for target knowledge vs. draft noise, and curriculum-training the loss.
- Three surgical components: 1. *Adaptive Layer Fusion*. A learnable weight matrix
- Headline results (greedy decoding, average speedup):
- Qwen3-4B: 5.52x (vs DFlash 4.99x, +11%)
- Qwen3-8B: 5.46x (vs DFlash 5.05x, +8%)
- GPT-OSS-20B: 3.91x (vs DFlash 3.71x, +5%)
- Largest gains on math reasoning (GSM8K, MATH500) and consistent gains on HumanEval, MBPP, MT-Bench.
- Scaling behavior unlocked:
- Draft depth 5 → 6 → 7 layers: DFlare's acceptance length and speedup rise monotonically; DFlash saturates immediately.
- Target layers used T = 5 → 7 → 9: DFlare keeps improving; DFlash plateaus at T=7 due to the single-FC bottleneck.
- Training data 270K → 800K → 2.4M: DFlare keeps improving with no visible saturation (authors flag larger-scale training as a promising next step).
- Ablations: Removing softmax normalization causes the largest drop (weight constraint is critical for stability); removing heterogeneous KV projections or the progressive loss also hurts; adding an extra FC after fusion yields no gain, confirming the lightweight design is sufficient.
- Stochastic decoding (T=1): Acceptance-length gains of +10.9% (Qwen3-4B), +11.3% (Qwen3-8B), +3.9% (GPT-OSS-20B) over DFlash.
- Engineering release: Integrated into Tencent's open-source AngelSlim toolkit; supports vLLM and SGLang backends. Recommended config:
num_draft_layers=7,block_size=16,num_target_layers=9,training_samples=2.4M. - Limitations: Higher one-time training cost (deeper draft + 3x more data); data-scaling curve not exhausted; behavior on 100B+ target models still to be verified.
W^fuse ∈ R^(D×T) produces a per-draft-layer softmax over T selected target layers. The fused feature is f_t^(i) = RMSNorm(Σ_j α_j^(i) · h_t^(j)). Only D×T scalars are added; for D=7, T=9 that is ~63 parameters. After training the weights are cached, so inference is a scalar weighted sum with zero added latency.
2. *Heterogeneous KV Projections*. Target context features use their own W_K^t, W_V^t; draft tokens and target decode tokens share W_K^d, W_V^d. Queries still come from draft positions, so target semantics and diffusion noise no longer compete for the same projection.
3. *Progressive Position-Weighted Loss*. A linear warmup of the decay parameter γ(s) = γ_0 + (s/S)·(γ_max − γ_0) gradually shifts optimization from early (easy) positions in the block to late (hard) ones, replacing DFlash's fixed exponential schedule.
Why it matters
DFlare shows that the draft-model capacity ceiling in Block Diffusion speculative decoding is not a parameter-count problem but an information-flow problem. With only ~63 extra parameters, layer-wise fusion re-opens scaling along draft depth, number of target layers, and training-data size simultaneously, delivering roughly 5.5x end-to-end speedups on widely used Qwen3 targets.