Diffusion language models write text differently from ordinary chatbots. Instead of producing one word at a time from left to right, they start from a fully blanked-out answer and fill in many blanks at once over a few rounds. Once a blank is filled, it usually stays filled. That makes them fast, but only if the model can tell which blanks are safe to fill early. Every fast sampler today makes that call from the model's own top-word probability. The same models are known to be badly overconfident: they assign more than 0.93 confidence to answers whose accuracy has collapsed. So the speed of these systems rests on the least reliable number they produce.
We propose to replace that number with a learned estimate of commit risk: the probability that writing a given word at a given position now will turn out wrong. The estimator is a small probe that reads hidden states the model already computes, together with how the word's probability has moved over recent rounds. It is trained once per model on cheap, automatically generated labels. Decoding then commits a word only when its estimated risk falls below a budget. Averaged over the words that make up the final answer, the same estimates also give an answer-level confidence that we expect to be far better calibrated than raw probabilities. The bet is that one inexpensive estimator improves both the speed–accuracy tradeoff and the honesty of the reported confidence.
Core Claim
The decision "commit this word in this round" is a prediction problem with its own correctness label. Raw softmax confidence is a poor and badly ordered predictor for it. A reviewer will note that any monotone recalibration of confidence (temperature scaling, isotonic regression) is equivalent to moving the commit threshold, and existing samplers already tune that threshold. The claim here is therefore specific. A commit-risk estimator that uses features beyond the token's own probability will reorder which positions get committed. We measure the payoff on two curves: accuracy against the number of model forward passes (which tracks wall-clock time), and the gap between stated confidence and actual correctness, measured by expected calibration error. The proposal stands or falls on whether this non-monotone reordering helps; the experiments isolate exactly that ingredient.
Concretely, at decoding round $t$, for a masked position $i$ with candidate token $\hat{y}_i$, let $h_i^{(t)}$ be the hidden states at a few chosen layers and $c_i^{(t-k:t)}$ the recent trajectory of top-1 probabilities. The probe $g_\phi$ outputs
$$r_i^{(t)} = g_\phi\big(h_i^{(t)},\, c_i^{(t-k:t)},\, \text{mask fraction},\, \text{distance to nearest committed token}\big) \approx P(\text{commit of } \hat{y}_i \text{ at round } t \text{ is wrong}).$$
Each round commits the lowest-risk positions whose cumulative risk stays under a per-round budget $\epsilon$, and always commits at least one position so decoding makes progress. Here $\epsilon$ is the single knob that trades speed for quality, in the same role as the confidence threshold in existing samplers.
Where This Sits in Prior Work
The motivating evidence comes from Beyond the Bidirectional Promise, which compares LLaDA-8B and Dream-7B with matched autoregressive models across 32 input-noise conditions. It finds that the diffusion models have higher calibration error and hold above 0.93 confidence on TriviaQA while accuracy collapses. It also shows that linear probes on their hidden states detect the corruption with over 93% accuracy, while the decoding loop fails to act on it. That paper diagnoses the problem but proposes no fix. Our probe is a direct attempt to route that internal signal into the commit decision. FAIR-Calib supplies a second constraint: under one-way unmasking, hidden representations keep moving for several rounds after a token is committed, which it calls "stability lag." FAIR-Calib uses this only to weight a quantization loss, and its "calibration" means quantization calibration, not confidence calibration. We turn the same observation into a decoding-time feature, measuring representation movement at the frontier.
The methods we compete with all gate commits on raw confidence. SlowFast Sampling commits high-confidence anchors and then decodes spans that have stabilized, under an explicit "Certainty Principle." AdaBlock-dLLM sizes blocks to confident punctuation, and its authors state that the method assumes calibrated confidence. PUNT ranks positions by confidence and then spends extra forward passes testing whether candidates influence each other. Their failure mode is the one Yadav et al. document: confident and stable is not the same as correct.
Our probe design borrows directly from PROBECAL in Uncertainty Calibration for Tool-Using Language Agents. PROBECAL trains a 3-layer MLP on hidden states of an autoregressive agent to predict task success, reducing ECE from 0.690 to 0.035 on TabMWP. That probe runs once per trajectory after generation. Ours runs at every position in every decoding round and controls the generation itself, which is a different and harder use. Large Language Models Must Be Taught to Know What They Don't Know supports learned confidence over prompted self-reports. It shows that verbal confidence fails on open-ended generation, and that roughly 1,000 graded examples suffice for a LoRA-tuned estimator to bring open-ended MMLU ECE from about 30% to 10%. That small data requirement is encouraging for our cheap labeling scheme.
Research Program
Labels. Commit-risk labels are generated automatically at two levels. The agreement label asks whether a token committed early under an aggressive parallel schedule matches the token at that position under a slow reference decode of the same prompt and model (one token per round, full step budget). This label isolates errors caused by parallelism and needs no gold answers, so it can be produced at scale on any prompt set. The correctness label asks, on tasks with checkable answers (GSM8K, TriviaQA, MBPP), whether a committed token belongs to a final answer that is wrong. This label lets the same probe speak to factual overconfidence, which slow decoding alone does not fix according to Yadav et al. The probe is trained on a mix of both labels. A separate head handles each label, so we can report which signal drives which gain.
Features and cost. Hidden states come from the forward pass the sampler already performs, so the probe adds one small MLP per masked position and no extra model calls. That contrasts with PUNT's $O(\log m)$ extra calls per round, where $m$ is the number of masked positions. Trajectory features (recent probability values and representation drift between rounds) are cached across rounds. We will report probe overhead in wall-clock time measured inside the dInfer and dLLM-Cache stacks, dInfer and dLLM-Cache, so that speed claims include caching effects.
Answer-level confidence. The final answer's confidence is one minus the mean commit risk over the tokens that make up the extracted answer. For agreement-trained heads we also report a min-based variant. We compare this against the mean token probability and against a PROBECAL-style probe applied once after generation.
Transfer. Probes are trained on one task family and tested on held-out tasks. They are also tested under the 32-condition input-noise protocol of Yadav et al., where raw confidence is known to fail worst. If a probe does not transfer, the method reduces to per-task tuning, and we will say so.
Baselines
The naive reference point is the model's default low-confidence-remasking sampler with a fixed number of tokens committed per round, run at several step budgets. It anchors the accuracy-versus-forward-pass curve and gives the uncorrected confidence ECE.
The strongest existing competitors are the confidence-gated parallel samplers. We run SlowFast Sampling from its released code, Slow-Fast-Sampling, plus AdaBlock-dLLM and a Fast-dLLM-style confidence threshold. All use LLaDA-8B-Instruct and Dream-7B, the same prompts, and the same answer extraction. For reference, AdaBlock-dLLM reports a 5.3% GSM8K accuracy gain over Fast-dLLM on LLaDA-Instruct, and SlowFast reports 54.75 tokens per second on LLaDA 8B against 33.79 for LLaMA-3 8B on an RTX 4090. We could find no reported calibration error for any of these samplers on Lacuna. Fairness comes from sweeping each method's own speed knob (threshold, block parameters, or our $\epsilon$) on a shared validation split with equal tuning budget. We then compare full accuracy-versus-forward-pass and accuracy-versus-wall-clock curves rather than single points. Beating these baselines means a curve that lies above theirs over a meaningful range of speeds. Concretely, we would need either equal accuracy at clearly fewer forward passes or higher accuracy at equal cost, holding across seeds on at least GSM8K and one code task.
PUNT is the strongest competitor on the coordination problem, since it explicitly tests whether parallel commits interfere. It reports up to 16% IFEval improvement over baseline unmasking schedules. We compare at matched total forward passes, counting its extra test calls. We also combine our risk ranking with its independence test to see whether the two signals are complementary.
Three ablations isolate the key ingredient. The first applies isotonic regression on top-1 confidence alone; this is a monotone map, so it should match the tuned-threshold baseline. The second adds trajectory features without hidden states. The third uses hidden states without trajectory features. If the full probe beats the confidence-only isotonic ablation, the gain comes from reordering, not from rescaling.
Why not just…?
Why not just temperature-scale the diffusion logits? Within a fixed threshold rule, a global temperature is a monotone change of confidence. It changes which threshold value means what, but it cannot change which positions look safest, so on the speed–accuracy curve it is equivalent to a threshold sweep. The isotonic ablation tests this directly. Temperature scaling can still improve answer-level ECE, so we include it as a calibration baseline. If temperature scaling matches the probe on ECE, the calibration half of our claim collapses to a known trick, and we would report that.
Why not just tune the confidence threshold or use variance tracking as SlowFast does? This is the real threat. Variance tracking catches tokens that are still fluctuating. However, Yadav et al. show models settling on stable, near-certain, wrong answers, and FAIR-Calib shows commits happening before representations settle. Both suggest that output stability lags behind and differs from correctness. The second ablation, which uses trajectory features without hidden states, is exactly the "smarter variance tracking" alternative. If it matches the full probe, the hidden-state component is unnecessary and the contribution shrinks to a learned stability rule.
Why not just fine-tune the diffusion model for calibration, as Kapoor et al. or Kong et al. do? Calibration fine-tuning, such as JSD-regularized LoRA in Kapoor et al. or manifold regularization in Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data, could make raw confidence trustworthy. If it did, existing samplers would benefit without any change. It changes model weights, however, and diffusion training is already high-variance according to Bringing Stability to Diffusion. It also targets answer correctness rather than errors introduced by parallel commits. We will run a LoRA calibration-tuned LLaDA under SlowFast and compare it against the frozen model with our probe at matched labeled-data budgets. If the tuned model closes the gap, that is a legitimate and simpler outcome.
Why not just let the sampler revise committed tokens instead of gating them? Remasking and revision attack the same irreversibility problem from the opposite side. Revision costs extra rounds and invalidates caches, and the file contains no reported revision results for us to compare against. Gating and revision can also combine, with the probe deciding what to revisit. We treat this as a follow-up and do not claim gating dominates revision.
Proposed Experiments
The main experiment draws accuracy-versus-forward-pass and accuracy-versus-wall-clock curves on GSM8K, MATH, HumanEval, MBPP and IFEval for LLaDA-8B-Instruct and Dream-7B, comparing the full probe against all baselines and ablations. The calibration experiment reports ECE and Brier score of answer-level confidence on TriviaQA, GSM8K and ARC-Challenge, under both clean inputs and the noise protocol of Yadav et al. The target is lower ECE than raw confidence, temperature scaling and a post-hoc PROBECAL probe, with no loss of accuracy.
Three checks decide the proposal. First, the full probe must beat the confidence-only isotonic ablation on the speed curve; otherwise the reordering idea is wrong. Second, under input noise the probe's risk must rise where raw confidence stays above 0.93. This is the test of whether the internal corruption signal Yadav et al. found can actually be routed into decoding. Third, a probe trained on math must keep most of its gain on code and instruction following; otherwise the method is per-task tuning dressed up as calibration. If all three hold, confidence-gated parallel decoding can rest on an estimate that is measured to be reliable.
References
- Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models
- FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
- Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
- AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
- Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
- Uncertainty Calibration for Tool-Using Language Agents
- Large Language Models Must Be Taught to Know What They Don’t Know
- Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data
- Bringing Stability to Diffusion: Decomposing and Reducing Variance of Training Masked Diffusion Models