On July 29, Tencent Hunyuan open-sourced AngelSpec, an end-to-end (training + deployment) speculative decoding framework, together with MTP/DFly draft model weights for Hy3-A21B. On Hy3-A21B, the DFly scheme achieves 1.98-2.40x end-to-end speedup over autoregressive decoding (tested at concurrency 4-64), with throughput 10.5-11.8% higher than DFlash, a popular contemporaneous work. Unlike DeepSeek DSpark (released 6/27), which focuses on the inference framework, AngelSpec builds draft model training + server-side deployment into one complete engineering pipeline, and supports MTP (Multi-Token Prediction) + block-parallel training. AI coding tool vendors can use it directly to cut response latency — per-token latency drops from 30-50ms to 15-25ms, a perceptible improvement for interactive code completion.
1. Where Speculative Decoding Sits in the Engineering Stack
Speculative decoding has become a standard optimization for AI coding tool vendors over the past 12 months. The principle:
1. A small draft model first generates k candidate tokens 2. The large target model verifies these k tokens in parallel 3. Correct tokens are accepted, wrong ones rejected — multiple tokens are produced in one "target model forward pass"
Engineering-wise, two bottlenecks have persisted: how to train the draft model and how to integrate it with inference engines. Traditional approaches — Medusa (extra FFN heads), Lookahead (Jacobi iteration), REST (retrieval-based) — each have drawbacks: Medusa requires modifying the main model architecture, Lookahead's speedup is unstable, and REST's hit rate depends on caches.
AngelSpec merges both concerns into a torch-native "training + deployment" framework that works out of the box.
2. The Core Numbers
Measured by Tencent Hunyuan on Hy3-A21B (Hunyuan 3 flagship, 295B/21B MoE):
- DFly: 1.98-2.40x end-to-end speedup vs autoregressive decoding, covering the common production concurrency range of 4-64
- DFly throughput 10.5-11.8% higher than DFlash — one of the most-watched speculative decoding works of late 2025, reused by Anthropic, Cursor, Codex, and others
- MTP (Multi-Token Prediction) + block-parallel training support — a training objective already used by GPT-4o-series and Llama 4, but rarely unified with serving deployment in open-source frameworks
- 6/27: DeepSeek DSpark speculative decoding framework open-sourced (decoding engine focus)
- 7/03: Alibaba DAMO Academy Elements Claw 1B superconductor materials discovery model + Agent framework
- 7/12: ByteDance Claude Code desktop app with built-in browser
- 7/23: Cursor Router cache-aware model routing + $4.63 per commit vs Fable 5's $12.69
- 7/29: Tencent Hunyuan AngelSpec (unified training + deployment framework)
- GitHub: https://github.com/Tencent/AngelSpec (training code)
- arXiv paper: https://arxiv.org/abs/2607.25852 (DFly algorithm details)
- Read the Docs: https://angelspec.readthedocs.io (deployment docs)
- Hugging Face: https://huggingface.co/collections/AngelSlim/angelspec (Hy3-A21B MTP/DFly draft weights)
- ModelScope: https://modelscope.cn/collections/AngelSlim/AngelSpec (domestic mirror)
- Chinese AI coding tools: per-commit inference cost can drop another 30-50% (stacked with Cursor Router's cache-aware routing)
- International AI coding tools: either adopt similar approaches or lose competitiveness on price/latency
- Underlying chips: speculative decoding is slightly less dependent on H100/H200 than autoregressive decoding (draft models can run on smaller GPUs); adapting AngelSpec to domestic chips (Huawei Ascend, Cambricon, Hygon) may be easier than running the main model — a hidden benefit for the domestic compute ecosystem
- Tencent Hunyuan X announcement: https://x.com/TencentHunyuan/status/2082447023626944936
- GitHub repository: https://github.com/Tencent/AngelSpec
- arXiv paper: https://arxiv.org/abs/2607.25852
- Hugging Face collection: https://huggingface.co/collections/AngelSlim/angelspec
- Read the Docs: https://angelspec.readthedocs.io
Notably, the AngelSpec paper (arxiv 2607.25852) also acknowledges TorchSpec from LightSeek Foundation — an open-source speculative decoding inference engine. Tencent Hunyuan positions AngelSpec as the "training side" and recommends pairing it with TorchSpec on the inference side; the two complement rather than compete with each other.
3. Concrete Value for AI Coding Tool Vendors
For vendors like Cursor, Claude Code, and Codex, the value is not "proving speculative decoding works again" (that consensus is settled) but three specific things:
1. Lower per-token latency: under autoregressive decoding, a 21B-active MoE model on A100/H100 has roughly 30-50ms per-token latency. A 1.98-2.40x speedup brings this to 15-25ms — an obvious experiential difference for interactive code completion. This latency is a core KPI in Cursor / Copilot devtool evaluations. 2. Draft model training is no longer a black box: previously, each company either trained its own draft model (expensive) or used Medusa-style approaches requiring architecture changes. AngelSpec ships complete training scripts plus directly downloadable Hy3-A21B MTP/DFly draft weights, eliminating in-house R&D cost. 3. Engineering the MTP training objective: MTP is the core training paradigm of Llama 4 and GPT-4o-series (predicting multiple future tokens at once, accelerating training and improving downstream tasks), but few open-source inference engines support MTP training. AngelSpec fills this gap — domestic AI coding tools can now more aggressively combine MTP training with speculative decoding.
4. Positioning Among Similar July Releases
Continuous output on the AI inference infrastructure side since July:
This wave of output from Chinese tech giants at the "AI inference infrastructure" layer forms a de facto standard candidate cluster: DeepSeek on inference engines, Alibaba on training paradigms + tooling, ByteDance on IDE integration, Tencent on speculative decoding training frameworks — none individually "disruptive," but together offering the Chinese AI coding ecosystem a path independent of the NVIDIA CUDA moat and OpenAI/Anthropic APIs.
5. Concrete Open-Source Moves
The release is complete, with five entry points:
Shipping weights + code + paper + docs + domestic mirror together is the "standard move" for Chinese open-source releases in 2026 H2 — minimizing the barrier to "plug and play," unlike some 2023-era releases that published papers without weights.
6. Potential Impact on Cursor / Anthropic / OpenAI
The essence of speculative decoding is squeezing more tokens out of each target-model forward pass; whoever has a more accurate draft model and tighter integration gets lower inference cost. Anthropic's Claude Fable 5 system reportedly uses speculative decoding internally (scheme undisclosed), and OpenAI GPT-5.6 Sol's disclosed "15%+ token generation efficiency gain" likely uses an in-house speculative decoding scheme.
If AngelSpec sees wide adoption by downstream tool vendors in 2026 H2: