DSpark: DeepSeek's Confidence-Scheduled Speculative Decoding Pushes Drafting From Quality to System Scheduling

Overview

DSpark is a speculative decoding system released by Peking University and DeepSeek. The paper, "DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation" (arXiv:2606.19348), is published under MIT license with reference code at github.com/deepseek-ai/DeepSpec and draft weights at huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark. It targets the fact that large Transformer inference is memory-bandwidth bound: for a 1.6T-parameter MoE such as DeepSeek-V4-Pro that activates 49B parameters per token, GPU compute utilization often stays below 10% because the device spends most of its time moving weights from HBM to SMs rather than computing.

Background: Why speculative decoding?

Speculative decoding uses a small model to draft several candidate tokens, then asks the large model to verify them in parallel. Because verifying a short sequence costs roughly the same as generating one token, high draft quality translates directly into throughput. Existing approaches split into two camps:

• Autoregressive drafts (e.g., Eagle3): high accuracy, slow draft generation.
• Parallel drafts (e.g., DFlash): fast drafting, but acceptance rate collapses as draft length grows because later tokens never see earlier sampled tokens — only their hidden states.

DSpark tries to take the best of both.

Design 1: Semi-autoregressive generation

DSpark stacks a heavy parallel backbone with a lightweight sequential head:

1. Parallel backbone produces hidden states for a draft block of length γ in one shot. 2. Markov head is a low-rank head that conditions each position on only the previous token, adding a left-to-right bias to the logits so later draft tokens can react to earlier ones.

Reported effects:

• Two DSpark layers outperform five layers of conventional parallel draft architectures.
• Offline acceptance length is 16–18% higher than DFlash and 26–31% higher than Eagle3.

A heavier RNN head variant is also discussed; it sees more prefix context and can score higher on some tasks, but adds latency. Production uses the Markov head.

Design 2: Confidence-scheduled verification

Traditional speculative decoding verifies a fixed γ tokens every step, regardless of draft quality. DSpark instead trains a small confidence head alongside the drafter that predicts, per token, the probability that the target model will accept it. A hardware-aware scheduler then uses this score:

• If confidence drops below a threshold, the draft is cut short.
• If GPU load is low, more tokens are verified.
• If GPU load is high, verification is kept conservative to protect concurrency.

Reported acceptance-rate gains from the scheduler:

• Chat workloads: 45.7% → 95.7%
• Math reasoning: 76.9% → 92.5%

Measured results on DeepSeek-V4

Single-stream generation speedups over an MTP-1 baseline:

| Model | Speedup | |--------|-------------| | V4-Flash | 60–85% | | V4-Pro | 57–78% |

Throughput gains under strict latency SLAs:

| Model | SLA | Throughput gain | |---------|--------------|-----------------| | V4-Flash | 80 tok/s | +51% | | V4-Flash | 120 tok/s | +661% | | V4-Pro | 35 tok/s | +52% | | V4-Pro | 50 tok/s | +406% |

The advantage widens under tighter SLAs, because conventional fixed-length verification cannot speculate aggressively without risking timeouts. Quality is unchanged: DSpark is a lossless speculative decoding variant that preserves the target model's output distribution exactly.

Why it matters

• Serving economics. Inference, not training, dominates LLM cost. A ~6x throughput gain on existing hardware is a major lever for unit economics.
• Engineering leverage. Each component (single-token Markov dependency, low-rank factorization, confidence gating, hardware-aware scheduling) reflects deliberate frugality rather than brute-force scale.
• Open ecosystem. MIT-licensed code, vLLM and SGLang integration, published checkpoints for DeepSeek-V4-Pro/Flash, and validation on Qwen3 and Gemma4 families extend the benefit beyond DeepSeek's own models.

Limitations and open questions

• Draft-model training cost (10 epochs on Open-PerfectBlend 2 per the paper); teams without training infrastructure should use the released checkpoints.
• No universal rule for Markov vs. RNN head; choice is workload-dependent.
• Compatibility assumes an autoregressive target model; non-AR architectures (e.g., block-diffusion) need adaptation.
• Million-token contexts are not addressed; KV-cache pressure on the drafter itself becomes a separate bottleneck (related work such as Mooncake targets the serving stack).

TL;DR

DSpark is not a new theoretical breakthrough but a careful engineering compromise between fast-but-noisy parallel drafts and accurate-but-slow autoregressive drafts: a Markov head injects just enough sequential dependency, and a confidence head prevents wasted verification. Result: up to 80% faster generation, up to 6x higher throughput, zero quality loss, and open source.

References

• Paper: DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation — arXiv:2606.19348
• Code: github.com/deepseek-ai/DeepSpec (MIT)
• Weights: huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark
• Deployment: vLLM, SGLang
• Related: DFlash (Chen et al., 2026), Eagle3 (Li et al., 2026), Mooncake (DeepSeek KV serving)