Faster Block-Diffusion Serving with Distribution-Free Risk Guarantees

Jungseob Lee, Dongyub Jude Lee, Chanjun Park, Sugyeong Eo, Heuiseok LimarXiv 2026 · arXiv

This paper introduces Redline, a statistically rigorous framework for serving block-diffusion language models (BD-LMs) with guaranteed reliability. By moving beyond simple "mean accuracy" metrics, the authors provide a way for engineers to aggressively speed up model serving while ensuring that the probability of the model failing on previously "easy" prompts stays within a strict, user-defined budget.

TL;DR

Redline is a distribution-free procedure that selects the fastest serving configuration for a language model while guaranteeing, with high confidence, that it won't regress on more than a fixed percentage of prompts compared to a reference model. It enables significant speedups—up to 46% more tokens per forward pass—across math and code tasks without the "silent failures" typical of manual tuning.

What Problem Does This Paper Address?

Serving modern language models, especially block-diffusion models that generate text in parallel chunks, involves tuning many "knobs": acceptance thresholds, buffer depths, and quantization levels. Traditionally, these are picked based on mean benchmark accuracy.

However, mean accuracy is a deceptive metric. A faster configuration might maintain the same average score by getting new hard prompts right while simultaneously failing on easy prompts the original model handled correctly. These "regressions" are frustrating for users and dangerous for production systems. The authors argue that operators need a risk budget: a way to say, "Give me the fastest possible settings, provided there's less than a 10% chance that this new version fails where the old one succeeded."

Methods

The authors propose Redline, a four-step procedure based on the "Learn-Then-Test" statistical framework:

  1. Define a Grid: Create a set of candidate serving configurations (varying thresholds, checkpoints, etc.).
  2. Score Regressions: Evaluate these on a calibration dataset, specifically counting "reference-relative risk"—instances where the reference model is correct but the candidate is not.
  3. Statistical Testing: Calculate a p-value for each configuration using an exact binomial test. This provides a finite-sample guarantee without making assumptions about the underlying data distribution.
  4. Multiplicity Correction: Use the Holm step-down procedure to account for testing multiple configurations at once, then deploy the fastest one that passes the risk test.

The paper also analyzes three "levers" within the serving loop to improve efficiency:

  • Training: Using self-distillation on engine-decoded targets to make the model more confident.
  • Allocation: A "static skip rule" that avoids wasting compute on blocks that are still fully masked.
  • Commitment: Adjusting thresholds to decide when a denoised token is "ready" to be finalized.
Figure 1: Levers inside the serving loop and risk control outside it. Top, the three levers at the decision sites they act on. Bottom, Redline tests every configuration on the correctness of its answers, keeps those that pass (green) and deploys the fastest.

Datasets

The researchers evaluated Redline using two block-diffusion model families (LLaDA2 and SDAR) across several standard benchmarks:

  • Math: GSM8K and MATH-500.
  • Code: HumanEval+ and a variant of MBPP.
  • Autoregressive Baselines: Llama-3.1-8B and Qwen2.5-7B were used to test Redline's applicability to speculative decoding and quantization.

Key Insights and Ideas

  • Mean Accuracy Hides Regressions: The authors demonstrate that a configuration can look "safe" on average while failing on nearly 7% of prompts the reference model got right. Redline explicitly bounds these regressions.
  • Math is "Safer" to Accelerate: Interestingly, math tasks gain speed at much lower risk budgets than code tasks. At a 10% risk budget, math models saw massive speedups, while code models required a slightly higher budget to see gains.
  • The "Two-Block" Ceiling: The authors discovered that in multi-block serving, only about two blocks typically stay "in flight." Deeper blocks remain too uncertain to commit, meaning simply increasing buffer size yields diminishing returns.
  • Generality: While designed for diffusion models, Redline works "out of the box" for other lossy accelerations like speculative decoding and weight quantization (e.g., selecting between int8, nf4, and fp4).

Key Results

  • Significant Speedups: For LLaDA2 on math tasks, Redline deployed a configuration that committed 37.5% more tokens per forward pass (TPF) at a 0.10 risk budget.
  • Synergy with Distillation: When combined with a self-distilled checkpoint, Redline achieved a 46.7% TPF gain on MATH-500, significantly outperforming hand-picked configurations.
  • Reliability: In 1,000 random data splits, Redline stayed within its stated failure probability, whereas traditional mean-accuracy rules frequently exceeded the risk budget.
  • Quantization Efficiency: Redline successfully identified nf4 as a valid quantization format for Llama-3.1-8B, reducing weight memory by 2.81x while staying within the risk bound.
Deployed TPF gain of Redline on MATH-500.

Limitations and Future Work

  • Calibration Sensitivity: The guarantees are relative to the calibration distribution. If the production prompt distribution shifts significantly, the model must be re-calibrated.
  • Verifiable Correctness: Redline requires a way to verify if an answer is "correct" (e.g., unit tests for code or exact matches for math). Applying this to subjective chat or creative writing would require a reliable automated judge or reward model.
  • Batching Effects: The authors note that different batching policies can act as a distribution shift, suggesting that re-calibration is necessary when changing serving infrastructure.

Summary updated

Abstract

Block-diffusion language models are served at hand-picked operating points, such as acceptance thresholds, buffer depth, schedule, checkpoint and precision, and each point is chosen by its mean benchmark accuracy. However, a mean does not tell an operator how often a faster configuration fails on prompts that the slower one answers correctly. On the serving engine and its decode traces, the default commit rule already commits every fully resolved block, a static skip rule captures nearly all of the compute that allocation can save, and self-distillation on engine-decoded targets adds speed at unchanged accuracy. Larger speedups come from lower thresholds, which commit tokens that are still uncertain. We therefore present Redline, a finite-sample procedure that selects operating points, hand-picked or learned, from the correctness of their answers on calibration prompts. Redline keeps the reference-relative risk, the joint probability that the reference answers correctly and a candidate configuration does not, within a user-chosen budget with high probability, and deploys the fastest configuration that passes. It speeds up math at a smaller risk budget than code in both model families, and at a budget of ten percent it deploys a LLaDA2 math configuration that commits over a third more tokens in each forward. It also applies without modification to the acceptance rule of speculative decoding and to weight quantization. On the same calibration data, Redline stays within its stated failure probability, whereas each tolerance of a mean-accuracy rule either gains less speed for some model and task or exceeds the risk budget far more often for another. Code is available at https://github.com/js-lee-AI/Redline.
BibTeX
@misc{lee2026faster,
  author = {Jungseob Lee and Dongyub Jude Lee and Chanjun Park and Sugyeong Eo and Heuiseok Lim},
  title = {{Faster Block-Diffusion Serving with Distribution-Free Risk Guarantees}},
  year = {2026},
  eprint = {2609.33887},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.33887},
}
Figure