The problem: autoregressive decoding wastes GPU capacity
Suppose you run a local large language model like Qwen3.5-27B. When it writes code, tokens come out one by one—this is autoregressive decoding: the next token can't be generated until the previous one is done. The GPU is powerful, but running serially like this leaves most of its compute idle.
Prior art: speculative decoding
The classic solution is Speculative Decoding: a small model quickly "guesses" a sequence of tokens, then the large model verifies them all in parallel. Accurate guesses save time. EAGLE-3 is a top method in this space, but its drafter is still autoregressive—guessing 8 tokens means 8 forward passes, so costs grow linearly. That forces shallow drafters that guess poorly.
DFlash's idea: use a diffusion model to draft
A diffusion model works like restoring a blurry photo: start from noise and denoise step by step. The key property is parallelism—one step can update an entire image at once.
DFlash applies this to text. It trains a small diffusion drafter that, whether guessing 8 or 16 tokens, completes the draft in a single forward pass. This means the drafter can be much deeper—a 5-layer Transformer versus EAGLE-3's 1 layer—guessing more accurately while having lower latency.
The core innovation: target-model features injected at every layer
A drafter guessing from scratch won't be accurate. DFlash extracts hidden-state features from the large model and injects them into every layer of the small drafter. The large model "knows" what might come next; the small model uses that information to guess. It's like getting turn-by-turn hints from a friend over a walkie-talkie at every intersection—not just at the start.
Results
- Qwen3.5-27B on HumanEval: 84 → 427 tokens per second, a 5.2x speedup
- Completely lossless: output quality is identical to the original model
- Very low training cost: the drafter reuses the target model's embedding and LM head (frozen); only the intermediate layers are trained
- One-step denoising: because of the target's hidden features as conditioning, the traditional multi-step diffusion iterations aren't needed
- Supported frameworks: vLLM, SGLang, Transformers
- The drafter is only 2B parameters for the 27B version—very lightweight
There's a subtle reframing here. People used to think diffusion models needed to catch up to autoregressive models on generation quality. DFlash asks a different question: does a diffusion model need to compete on quality at all—or just be an excellent guesser? The answer: it just needs to guess, quickly and accurately; verification is left to the autoregressive model. Each does what it's best at.
Explaining it to a freshman
A normal LLM writes carefully, one character at a time. DFlash uses a fast "draft generator"—quick but not always right—then the large model, like a teacher grading homework, checks the draft, keeping correct parts and fixing wrong ones. The drafter uses a diffusion model, so it writes a whole block at once, and it peeks at the large model's "thoughts" to guess better.