Key points
> Paper: *DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation* > Institutions: Peking University + DeepSeek | Open source: MIT (github.com/deepseek-ai/DeepSpec) > arXiv: 2606.19348 | Released: 2026-06-27 > TL;DR: A Markov head eliminates the "suffix decay" of parallel drafting, while a confidence head schedules verification length on demand — inference speed +80%, throughput +600%, lossless.
Why is LLM generation so slow?
Autoregressive Transformers emit one token at a time. Even with KV caching, the bottleneck is memory bandwidth: for a giant MoE model like DeepSeek-V4 Pro (1.6T parameters, 49B activated per token), the GPU mostly waits on moving weights from HBM — it is memory-bound, not compute-bound, often with under 10% compute utilization.
Speculative decoding tackles this: a small draft model quickly generates a block of candidate tokens, and the large model verifies them in parallel (verifying a block costs roughly the same as generating one token). Higher draft quality → more accepted tokens → bigger speedup. The catch: draft quality decays along the sequence (e.g., ~95% acceptance for token 1, below 50% after token 5), because parallel drafting cannot see the concrete values of earlier drafted tokens.
Existing trade-off:
| Approach | Representative | Pros | Cons | |---|---|---|---| | Autoregressive drafting | Eagle3 | Accurate, slow decay | Drafting itself is slow (serial) | | Parallel drafting | DFlash | Fast (whole block at once) | Severe suffix decay, long drafts wasted |
Design 1: Semi-Autoregressive Generation
DSpark pairs a parallel backbone (generates hidden states for a full γ-token draft block at once) with a lightweight Markov head that conditions each position only on the previous token, via low-rank decomposition, adding a prefix-dependent bias to the logits. The backbone provides speed; the Markov head provides accuracy.
- A 2-layer DSpark module beats 5-layer traditional parallel draft architectures
- Offline acceptance length: +16-18% over DFlash, +26-31% over Eagle3
- An RNN-head variant (more prefix history) performs better on some tasks but adds latency; production uses the Markov head
- Terminates the draft early when confidence drops below a threshold
- Verifies more tokens when GPU load is low
- Stays conservative at peak load to protect concurrency
- Draft model training cost: trained 10 epochs on Open-PerfectBlend 2; teams without training infrastructure can just use the released checkpoints.
- Markov vs. RNN head trade-off: no universal selection criterion; tune per task.
- Autoregressive assumption: needs adaptation for diffusion or other non-autoregressive architectures.
- Long-context scaling: at million-token contexts, the draft model's own KV cache becomes a bottleneck (not addressed here; see DeepSeek's Mooncake for KV serving).
- Paper: DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation (arXiv:2606.19348)
- Code: github.com/deepseek-ai/DeepSpec (MIT)
- Weights: huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark
- Deployment: vLLM, SGLang
- Related: DFlash (Chen et al., 2026), Eagle3 (Li et al., 2026), Mooncake (DeepSeek KV serving)
Design 2: Confidence-Scheduled Verification
Traditional verification blindly checks a fixed γ tokens regardless of draft quality. DSpark trains a confidence head (a small binary classifier) that predicts, per draft token, the probability the target model will accept it. A hardware-aware scheduler then:
Results: acceptance rate improved from 45.7% → 95.7% on chat, and 76.9% → 92.5% on math reasoning.
Measured results (production, DeepSeek-V4)
Single-user speed:
| Model | Speedup | Baseline | |---|---|---| | V4-Flash | 60-85% | MTP-1 | | V4-Pro | 57-78% | MTP-1 |
Throughput under strict SLA:
| Model | SLA | Throughput gain | |---|---|---| | V4-Flash | 80 tok/s | +51% | | V4-Flash | 120 tok/s | +661% | | V4-Pro | 35 tok/s | +52% | | V4-Pro | 50 tok/s | +406% |
Notably, the stricter the SLA, the bigger DSpark's advantage — confidence scheduling makes speculative decoding viable exactly where fixed-length verification would time out. Quality is lossless: DSpark is an exact speculative decoding variant preserving the target distribution.
Why it matters
1. Serving, not training, dominates inference cost. DSpark is a serving-layer optimization: ~6x serving efficiency means 6x users on the same hardware, or ~80% GPU cost savings. 2. Lean engineering. Every design choice minimizes waste: minimal-dependency Markov head, low-rank compression, on-demand scheduling, GPU-load-aware verification. 3. Open ecosystem. MIT-licensed code and weights, vLLM/SGLang integration, and generality beyond DeepSeek models (tested on Qwen3 and Gemma4 series).
Limitations
Bottom line
DSpark's contribution is not a new theory but an engineering sweet spot between "fast but rough" parallel drafting and "accurate but slow" autoregressive drafting: a Markov head adds just enough sequential dependency, and a confidence head avoids useless verification. Result: +80% inference speed, +600% throughput, zero quality loss, open sourced.
References