DARTree: Speculative Diffusion Decoding with Autoregressive Draft Tree Pruning

Forum topic · 小凯 · 2026-08-15

Summary

DARTree is a training-free speculative decoding method for autoregressive language models that extends a pretrained autoregressive (AR) correction head from single draft chains to trees. The method first constructs a fixed-width candidate tree by expanding and scoring all nodes at each depth in a single batch, then applies best-first pruning to select the verification tree, decoupling AR-head inference from sequential heap operations. This addresses the limitation of diffusion-based drafters, whose position-wise distributions are marginal rather than conditioned on tokens selected along each draft path, and of recurrent correction methods that only follow a single chain. Across seven math, code, and chat benchmarks, DARTree achieves the highest average acceptance length and speedup in all four model-temperature configurations, accepting up to 12.97 tokens per verification round—98.6% more than DFlash and 27.9% more than Domino—and delivering up to 9.73x lossless speedup over locally measured autoregressive decoding.

Overview

Field: Machine Learning Authors: Tianyi Li, Yaxin Luo, Xinyi Shang, Zhiqiang Shen Published: 2026-08-13 arXiv: 2608.13524

Abstract

Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path. Existing recurrent correction incorporates causal information along a single draft chain, whereas diffusion-based tree construction broadens candidate coverage without carrying this correction along individual branches.

We introduce DARTree, a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees. DARTree first constructs a fixed-width candidate tree by expanding and scoring all nodes at each depth in a single batch, and then only applies best-first pruning to select the verification tree, decoupling AR-head inference from sequential heap operations.

Results

Across seven math, code, and chat benchmarks, DARTree achieves the highest average acceptance length and speedup in all four model-temperature configurations:

Accepts up to 12.97 tokens per verification round
98.6% more accepted tokens than DFlash in the same setting
27.9% more accepted tokens than Domino in the same setting
Up to 9.73x lossless speedup over locally measured autoregressive decoding