D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding

Baoyou Chen, Ruiqiao Mei, Guancong Lin, Tao Song, Weihao Yuan, Qun Liu, Siyu ZhuarXiv 2026 · arXiv

TL;DR

D3D (Decoupled Diffusion Drafting) is a standalone diffusion-based draft model that enables asynchronous speculative decoding. By removing the dependency on the target model's hidden states, D3D can draft future token blocks while the target model is still verifying the current ones, achieving massive speedups—up to 7.48× on mathematical reasoning tasks.

What Problem Does This Paper Address?

Autoregressive language models are notoriously slow because they generate text one token at a time. Speculative decoding speeds this up by using a small "drafter" to guess several future tokens, which a large "target" model then verifies in parallel.

However, most state-of-the-art drafters are "target-conditioned," meaning they need the internal hidden states of the big model to make their next set of guesses. This forces a sequential bottleneck: the drafter must wait for the target model to finish verification before it can start drafting the next block. D3D addresses this by creating a standalone drafter that is accurate enough to work independently, allowing drafting and verification to happen at the same time (asynchronously).

Methods

The authors introduce three key innovations to make standalone diffusion drafting viable:

  1. Target-Aligned Drafter Training: Unlike previous diffusion drafters that rely on target features, D3D is trained to encode the token prefix using its own representations. It uses a dual-objective loss:
  2. Blockwise Denoising ($\mathcal{L}_{\text{diff}}$): Standard diffusion training to predict multiple tokens in parallel.
  3. Auxiliary Autoregressive Supervision ($\mathcal{L}_{\text{AR}}$): A standard next-token prediction loss that helps the drafter maintain strong language modeling capabilities and better understand the context prefix.
  4. Progressive Layer Pruning: To minimize drafting latency, the authors iteratively remove redundant layers. They identify layers whose removal has the least impact on the "acceptance length" (how many tokens the target model actually accepts) and perform recovery training via knowledge distillation from the original unpruned model.
  5. Asynchronous Inference: During decoding, D3D prepares multiple "successor blocks" in a single packed pass. It predicts what to do if the current block is fully accepted, or if it is rejected at various positions. This "shotgun" approach ensures that the next draft is often already waiting in a cache when the target model finishes verification.

Datasets

The model was trained on Open-PerfectBlend (1.42 million instruction-tuning examples). Evaluation was conducted across seven diverse benchmarks:

  • Math: GSM8K, MATH-500, AIME25.
  • Code: HumanEval, MBPP, LiveCodeBench-v6.
  • Chat: MT-Bench.
  • Long Context: 18 datasets from the LongBench suite.

Key Insights and Ideas

  • Independence is the Key to Speed: The "aha moment" is that a drafter doesn't need to be perfect; it just needs to be "good enough" and independent. By decoupling from the target model's hidden states, you can hide the drafting latency entirely behind the target model's verification time.
  • AR Loss is a Force Multiplier: Adding a simple autoregressive loss during diffusion training significantly improves the drafter's ability to represent the prefix, leading to longer accepted sequences without needing "hints" from the target model.
  • Drafters are Over-Parameterized: The pruning experiments showed that removing ~14% of the drafter's layers (5 out of 36) retained over 99% of the performance while providing a measurable speed boost.
  • Packed Successor Prediction: Instead of guessing one path, D3D guesses many. By using a "packed" forward pass to predict multiple possible outcomes of the current verification round, the system maximizes the chance of a "cache hit" for the next round.

Key Results

D3D consistently outperforms existing synchronous and asynchronous baselines (like DFlash, PARD-2, and SSD):

  • Average Speedup: Achieved 5.00× speedup with a Qwen3-8B target and 5.71× with a Qwen3-32B target.
  • Peak Performance: Reached a staggering 7.48× speedup on the GSM8K math benchmark.
  • Acceptance Length: D3D achieved an average acceptance length ($\tau$) of 7.88 tokens per round, significantly higher than DFlash's 5.96 at the same block size.
  • Efficiency: Pruning the model from 36 to 31 layers improved the mean speedup from 4.44× to 4.69× with negligible loss in quality.

Limitations and Future Work

  • Hardware Requirements: To fully realize the benefits of asynchronous execution, the system ideally requires two GPUs (one for the drafter and one for the target model).
  • Cache Miss Latency: If the target model rejects the draft in a way the drafter didn't anticipate (a "cache miss"), the system must fall back to synchronous drafting, losing the asynchronous advantage for that round.
  • Serving Integration: The current implementation uses the Transformers backend. Integrating D3D into high-throughput frameworks like vLLM or SGLang, which use complex batching and scheduling, remains an open engineering challenge.

Summary updated

Abstract

Target-conditioned diffusion drafters predict candidate tokens in parallel, yet each new block must wait for updated target-model hidden states. This forces drafting and verification to run sequentially, limiting speedup even though each block is generated in one pass. We introduce D3D, a standalone diffusion drafter that removes this inter-round dependency. D3D combines blockwise denoising with auxiliary autoregressive supervision to achieve long accepted prefixes without target hidden states. This independence lets D3D draft successor blocks during verification in one packed pass, enabling asynchronous speculative decoding. We progressively prune layers according to their effect on acceptance length to reduce the cost of packed drafting. Removing five of 36 layers retains over 99\% of mean acceptance length across three benchmarks with Qwen3-8B as the target. Across seven benchmarks, D3D averages 5.00$\times$ and 5.71$\times$ decoding speedup with Qwen3-8B and Qwen3-32B targets, compared with 4.03$\times$ and 3.67$\times$ for DFlash. On GSM8K with Qwen3-32B, speedup reaches 7.48$\times$. Code is publicly available at https://github.com/fudan-generative-vision/D3D.
BibTeX
@misc{chen2026d,
  author = {Baoyou Chen and Ruiqiao Mei and Guancong Lin and Tao Song and Weihao Yuan and Qun Liu and Siyu Zhu},
  title = {{D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding}},
  year = {2026},
  eprint = {2611.06425},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06425},
}
Figure