Schedule-Calibrated Commitment Order in Blockwise Masked Diffusion Language Models

Gaurav Sarkar, Umang Pandey, Pratik Prabhanjan Brahma, Zicheng Liu, Emad BarsoumarXiv 2026 · arXiv

TL;DR

When masked diffusion language models generate text in blocks, the resulting "left-to-right" commitment order is often a mathematical artifact of the schedule rather than a model-intrinsic property. This paper introduces a Spearman-rank calibration method to strip away these schedule-induced effects, revealing that the model's actual local ordering preference is weaker than raw global statistics suggest.

What Problem Does This Paper Address?

Masked Diffusion Language Models (MDLMs) like LLaDA offer a promising alternative to standard autoregressive models because they can predict multiple tokens in parallel. However, to maintain coherence, researchers often use blockwise inference, where the model generates a contiguous block of tokens (say, 64) before moving to the next.

When you plot which tokens are "committed" (finalized) over time, you often see a beautiful diagonal wave moving from left to right. It’s tempting to conclude that the model has "learned" to be autoregressive. But there’s a catch: if the schedule forces block 1 to finish before block 2 starts, a high global correlation is inevitable.

The authors ask: How much of that left-to-right "wave" is actually the model’s choice, and how much is just a consequence of the block schedule? Without answering this, we risk over-interpreting the model's internal mechanics.

Methods

The authors provide a "schedule-calibrated" interpretation of the Spearman-rank correlation ($\rho$), a standard metric for measuring how much the commitment order matches the positional order.

They derive a deterministic identity that decomposes the global correlation into two parts:

  1. Schedule Reference ($B_\sigma$): The correlation you would get if the model picked tokens within each block in a completely random order.
  2. Weighted Local Correlation ($W_\sigma \bar{r}_\sigma$): The actual contribution of the model's internal preferences within those blocks.

By calculating $B_\sigma$ and the weight $W_\sigma$ analytically based on the total sequence length ($N$) and block size ($\sigma$), they can "calibrate" the observed global correlation to find the mean within-block correlation ($\bar{r}_\sigma$). This value represents the model's true preference for left-to-right ordering within the decisions it was actually allowed to make.

Datasets

The study performs a secondary analysis on aggregate results from a paired study of LLaDA-8B-Instruct. The original study used 500 prompts, consisting of 125 prompts each from four popular benchmarks:

  • GSM8K and MATH (Mathematical reasoning)
  • MBPP and HumanEval (Code generation)

The evaluation fixed the generated length at $N=256$ tokens and compared two block sizes: $\sigma=64$ and $\sigma=128$.

Key Insights and Ideas

  • The Schedule "Floor" is Surprisingly High: For a sequence of 256 tokens split into four blocks of 64, even if the model committed tokens in exactly reverse order within every block, the global correlation would still be ~0.875. A "high" correlation doesn't necessarily mean the model is behaving autoregressively.
  • Raw Residuals are Deceptive: As block size increases, the "raw residual" (the difference between measured correlation and the random-schedule reference) often grows. One might think this means the model is becoming more left-to-right at larger scales. However, the authors show this is often just because the "weight" of the local order increases with block size.
  • Calibration Flips the Narrative: In their study, increasing the block size from 64 to 128 caused the global correlation to drop. While the raw residual increased, the calibrated local correlation actually decreased (from 0.751 to 0.660). This suggests the model's local ordering preference actually weakens as it is given more freedom.
Figure 1: Reported mean global commitment correlations for the same 500 prompts and the analytic uniform- random within-block references. Error bars reproduce the reported 95% prompt-bootstrap intervals; the interval at σ = 64 is smaller than the marker. Lines connect only the two evaluated schedules and do not imply a measured trend at intermediate block sizes.

Key Results

The paper's quantitative findings (summarized in Table 1) illustrate the power of calibration:

  • At $\sigma=64$: The global mean correlation was a near-perfect 0.984. However, the schedule reference was already 0.938. The calibrated local correlation was 0.751.
  • At $\sigma=128$: The global mean dropped to 0.915. The schedule reference dropped significantly to 0.750. Even though the "gap" between the measured value and the reference looked larger, the calibrated local correlation was lower at 0.660.
Table 1: Global summaries retained from the original study and algebraically calibrated quantities. Brackets give the reported 95% global interval and its affine transformations for the residual and local correlation. The reference and weight are deterministic, not estimated. Transformed intervals use the exact constants and rounded reported endpoints; no new resampling is performed.

The authors also performed a sensitivity check (Table 2) excluding terminal "End of Sentence" (EOS) tokens, confirming that these results aren't just driven by the model finishing the sequence early.

Limitations and Future Work

  • Empirical Scope: The analysis is limited to one specific model checkpoint (LLaDA-8B) and two specific schedules. It does not evaluate whether these ordering preferences correlate with better output quality or faster inference.
  • Schedule Assumptions: The current math assumes equal-sized, non-overlapping, contiguous blocks. It doesn't yet cover more complex "sliding window" or adaptive block schedules.
  • Data Constraints: The authors note that they worked with aggregate summaries rather than raw per-prompt commitment sequences, which limited their ability to perform new bootstrap resampling or per-dataset breakdowns.
  • Future Directions: The authors suggest that future commitment-order reports should always include $N, \sigma, B_\sigma, W_\sigma,$ and the calibrated $\bar{r}_\sigma$ to ensure results are interpretable and not just reflections of the chosen decoding schedule.

Summary updated

Abstract

Sequential block schedules can make token commitment in masked diffusion language models appear strongly left-to-right even when within-block order is random or reversed. For equal contiguous blocks processed without interleaving in positional order, with untied irreversible commitments, we use a Spearman-rank identity to express global position-commitment correlation as a deterministic schedule reference plus a weighted mean within-block correlation. In a reported paired study of LLaDA-8B-Instruct on 500 math and code prompts with 256 generated positions and updates, increasing block size from 64 to 128 lowers mean global correlation from 0.9845 to 0.9149. Uniform-random within-block references are already 0.9375 and 0.7500. After calibration, mean within-block correlations are 0.751 and 0.660, so the larger raw residual at block size 128 does not indicate stronger local positional ordering. We give sharp bounds and derive calibrated intervals from reported summaries, while documenting the limitations of the aggregate-only empirical release. The analysis supports schedule-aware interpretation of commitment order; its empirical scope is one checkpoint and two schedules, without evaluation of output quality or inference speed.
BibTeX
@misc{sarkar2026schedule,
  author = {Gaurav Sarkar and Umang Pandey and Pratik Prabhanjan Brahma and Zicheng Liu and Emad Barsoum},
  title = {{Schedule-Calibrated Commitment Order in Blockwise Masked Diffusion Language Models}},
  year = {2026},
  eprint = {2611.15234},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15234},
}
Figure