Learning the Decoding Schedule of Diffusion Language Models by Unrolling It

Diffusion language models are text generators that start from a fully masked sequence and fill it in over many rounds, revising their guesses at every round. They produce text by iterative refinement, and their latency is set by how many rounds they run and how cautiously each round commits to tokens. Today those choices are made by hand. Engineers pick how many rounds to run, how to blend a "still unknown" placeholder with the model's current best guess, and when to stop and lock in tokens. We propose to treat the decoding loop of a frozen diffusion language model as an optimization algorithm and to unroll it. Unrolling means writing each round as one layer of a network and training the small set of per-round settings end to end by backpropagation, the same way unrolled sparse-coding and primal-dual solvers learn step sizes that let them finish in a handful of iterations instead of hundreds.

The core claim is that a tiny learned controller can replace the hand-tuned blending and stopping rules. The controller has a few thousand parameters, reads the model's uncertainty at each round, and outputs a step size and a commit probability for every position. If it is trained so that the output is good at every truncation depth, a single model gives a dial between latency and accuracy. That dial should dominate tuned heuristics at each budget. If the claim holds, the decoding schedule becomes something learned rather than engineered, without touching the backbone weights.

What exists and what is missing

The closest work is Latent Refinement Decoding (LRD). LRD replaces hard [MASK] tokens in models like Dream and LLaDA with entropy-weighted mixtures of mask vectors and expected token embeddings, refines these soft "belief states," and stops early when the KL divergence between consecutive rounds falls below a threshold. It reports up to a 10.6x speedup on 1024-token decoding with Dream-Ins-7B. LRD supplies our state representation. Its mixture weights are a fixed entropy formula, its stopping rule is an empirical threshold, and nothing in the loop is trained. We keep its soft belief states and learn everything LRD hand-sets.

Two other lines collapse refinement instead of controlling it. DCFold distills AlphaFold3's recycling and diffusion loops into one forward pass, which yields a 15x speedup. The cost is a drop from 82.9% to 78.6% success on PoseBusters V2 and an acknowledged narrowing of output diversity. Meta-Reflection replaces multi-turn textual reflection with codebook vectors injected at a hand-chosen intermediate layer. Both give one operating point, not a dial. Text-space approaches such as Parallel-Distill-Refine and Self-Refinement Tuning pay for every revision in generated tokens. They motivate the latency problem but operate in a different regime from ours.

The method comes from algorithm unrolling. Cherabier et al. learn iteration-specific parameters of a primal-dual solver and converge about 100x faster than the classical variational method. UNEM unrolls generalized EM and learns its temperature and class-balance hyperparameters in place of grid search. That is the closest analogue to replacing LRD's tuned thresholds. A constraint comes from Chang et al., who show that backpropagating through many unrolled refinement steps explodes gradients and costs memory linear in depth. Their fix, implicit differentiation at a fixed point, does not transfer directly here. Decoding deliberately commits tokens and is not run to a fixed point, so we must handle the gradient problem differently.

Core Claim

Let $b_k$ be the per-position belief over the vocabulary at round $k$, and let $f$ be the frozen diffusion backbone applied to the soft embedding of $b_k$. One unrolled round is

$$b_{k+1} = (1-\alpha_k)\, b_k + \alpha_k\, \mathrm{softmax}\big(f(b_k)/\tau_k\big),$$

where $\alpha_k \in (0,1)$ is a per-position step size and $\tau_k$ is a temperature. Both are output by a small controller $g_\phi$. The controller sees each position's entropy, the change in belief since the last round, and the round index, and it also emits a commit probability that relaxes LRD's soft-to-hard switch. We train $\phi$ with the backbone frozen to minimize

$$\mathcal{L}(\phi) = \mathbb{E}_{K \sim p(K)}\; \mathrm{CE}\big(b_K,\, y^\ast\big),$$

where $y^\ast$ is a correctness-filtered output of the same backbone run with its long default schedule (for example 256 rounds), and $K$ is drawn from horizons such as 2 to 32. Sampling the horizon is what makes the solver anytime: no single $K$ is privileged, so the controller must produce a usable answer whenever it is cut off. Because the loss uses the model's own verified long-schedule outputs, the question is purely whether a learned schedule can reach the same answers in fewer rounds. It is not a fine-tuning study.

Research Program

We will build the controller on Dream-7B and LLaDA, starting from LRD's soft-state code path and the diffusion reasoning tooling in diffusion-of-thoughts. Training needs gradients through up to 32 backbone calls. We will use gradient checkpointing per round and compare three ways of computing gradients. The first is full backpropagation through time. The second truncates backpropagation to the last $m$ rounds. The third is a one-step gradient in the spirit of Chang et al.'s first-order approximation, applied per round rather than at a fixed point. This comparison tests directly whether their instability finding carries over to frozen-backbone decoding. We expect it to matter less here, because the backbone weights get no gradient and the controller's outputs are bounded.

The training set is GSM8K and MBPP problems. We test transfer to MATH500 and HumanEval without retraining, so the controller cannot simply memorize task-specific thresholds.

Baselines

The naive reference point is the backbone's default confidence-based unmasking, run at reduced round counts (2, 4, 8, 16, 32) with its standard settings. It shows how much accuracy is lost by cutting rounds with no intelligence at all. The theory in Adaptation to Intrinsic Dependence in Diffusion Language Models, which shows that randomized unmasking can decode in fewer rounds than tokens, suggests this baseline may be stronger than expected, so we include a randomized-schedule variant.

The strongest existing method is LRD. The file reports its gains for Dream-Base-7B as +2.9 on GSM8K, +3.8 on MATH500, +6.3 Pass@1 on HumanEval, and +2.6 on MBPP, with up to a 10.6x speedup on Dream-Ins-7B. No absolute accuracies are reported on Lacuna for LRD, so we will reproduce it ourselves. For fairness, LRD's free settings are its entropy weighting, its KL threshold, and its soft-to-hard transition point. We will tune these separately at every round budget with Bayesian optimization on the same GSM8K and MBPP training data, using the same number of backbone evaluations that training our controller consumes. Beating LRD means higher accuracy at matched backbone calls and matched wall-clock on all four benchmarks, at budgets of 8 rounds or fewer, from one controller evaluated at every budget, against per-budget-tuned LRD.

Two ablations isolate the key ingredients. The first is a global learned schedule: one $\alpha_k, \tau_k$ per round, shared across positions and inputs. It separates the benefit of learning from the benefit of conditioning on the current belief state. The second is a controller trained at a single fixed $K$, which tests whether the sampled-horizon loss is what makes the dial work.

Why not just…?

Why not just tune LRD's hyperparameters per budget? This is the most serious threat. The controller has few parameters, and if all of its gain comes from finding better global constants, Bayesian optimization would find them too. The global-schedule ablation and the per-budget-tuned LRD baseline exist to settle this. If the learned global schedule matches tuned LRD and the state-conditioned controller adds nothing, we will report that unrolling buys convenience but not accuracy. That outcome would itself be a useful negative finding for the LRD line.

Why not just distill the model into a few-step student, as DCFold does? Distillation can reach a single low-latency point and may beat us there. It gives up the dial, and DCFold's own report of concentrated outputs suggests it costs diversity. We will compare against a few-step student distilled from the same long-schedule outputs. The comparison is accuracy at its native budget plus pass@k on HumanEval and MBPP, which measures diversity, and we will state plainly if the student wins at its operating point.

Why not just run plain backpropagation through the unrolled loop? That is one of our three gradient variants, not an alternative to the proposal. Chang et al. found that vanilla unrolled SLATE reached 67.04 reconstruction MSE on CLEVR-Mirror versus 10.66 with implicit gradients. If full backpropagation is stable here because the backbone is frozen, we will use it. If it is not, the truncated and one-step variants are the fallback.

Decisive Checks

The central experiment plots accuracy against backbone calls and wall-clock for every method on GSM8K, MATH500, HumanEval, and MBPP. The claim holds only if the single anytime controller lies on or above every per-budget-tuned baseline curve. It must also do so on the held-out benchmarks, not only on the training tasks.

A second check asks whether the learned step sizes behave as expected. They should shrink on high-entropy positions and on harder problems. Measured as rounds used per problem when the controller's commit probabilities trigger early stopping, they should correlate with problem difficulty. That would confirm that the controller allocates rounds to hard inputs rather than learning a constant.

The failure mode we will look for is semantic drift at very small $K$. Committed tokens could become fluent but wrong, so for code we will check whether failures are execution errors or wrong logic, measuring execution with mxeval.