Suppose you're running a local LLM, such as Qwen3.5-27B. When it writes code, tokens come out one at a time — that's autoregression: the next token can only be generated after the previous one finishes. GPU compute is powerful, but running serially like this means most of the time is spent waiting, and utilization is pitifully low.
A previous solution is Speculative Decoding: a small model quickly "guesses" a block of tokens, and the large model checks in parallel which guesses were right. Accurate guesses save time. EAGLE-3 is a top method in this direction, but its drafter is also autoregressive — guessing 8 tokens requires 8 passes, so cost grows linearly. As a result, the drafter can only be shallow and guesses are inaccurate.
DFlash takes a different approach: it uses a diffusion model to guess.
What's a diffusion model? Imagine a blurry photo; the diffusion process gradually sharpens it. In reverse, starting from noise and denoising step by step yields a clear image. The key point: denoising is parallel — a single step can affect every pixel of the whole image.
DFlash applies this idea to text. It trains a small diffusion model (the drafter) whose only job is "guessing." Whether it drafts 8 or 16 tokens, it's done in a single forward pass. This means the drafter can be deep — a 5-layer Transformer, much stronger than EAGLE-3's 1 layer — guessing more accurately with lower latency.
But that's not enough. If the small model guessed entirely from scratch, it would be inaccurate. DFlash's core innovation: it extracts hidden features from the target model and injects them into every layer of the small model. The large model "knows" what might come next, and the small model uses that information to guess.
Analogy: imagine walking down an unfamiliar road. Walking blind, you'd probably take wrong turns. But with a walkie-talkie, a friend ahead tells you "turn left at the next intersection," and your odds improve dramatically. DFlash doesn't just hint at the starting point — it hints at every intersection.
Measured results: Qwen3.5-27B on HumanEval went from 84 tokens per second to 427 tokens per second with DFlash — a 5.2x speedup. And it's completely lossless: output quality is identical to the original.
This reminds me of something. People used to believe diffusion models couldn't match autoregressive models in generation quality, so the goal was helping diffusion "catch up." But DFlash asked a different question: does a diffusion model really need to compete on quality? Or does it just need to be an excellent "guesser"?
The answer now seems obvious — it only needs to guess. Guess fast, guess accurately, and leave verification to the autoregressive model. Each plays to its strengths.
Is that real understanding? Let me test it. If you had to explain DFlash to a college freshman, what would you say?
You'd say: imagine writing a long sentence. A normal LLM is like thinking carefully, one character at a time. DFlash instead uses a "drafter" to quickly produce a draft — fast, but not necessarily correct. The large model then checks it like a teacher grading homework: keep what's right, fix what's wrong. The drafter uses a technique called a "diffusion model," which writes fast (a whole chunk of characters at once) and peeks at the large model's "thoughts" to guess more accurately.
Did that make sense? If so, my understanding is probably okay. If it's still confusing, maybe I haven't grasped it myself.
That's DFlash. No magic — just combining two things so each does what it's best at. Clever, but not complicated.
---
Additional technical details:
- Extremely low training cost: the drafter reuses the target's embedding and LM head (frozen); only intermediate layers are trained
- One-step denoising: because target hidden features serve as conditioning, the traditional multi-step diffusion iterations aren't needed
- Adapted frameworks: vLLM, SGLang, Transformers
- The drafter is only 2B parameters (for the 27B version) — very lightweight