Learned Commitment Order for Multimodal Diffusion Reasoning

When a vision-language model reasons step by step, an early mistake such as a miscounted object or a wrong color becomes a premise it never revisits, and later steps stay consistent with it rather than with the image. A masked diffusion model offers a different route. It is a generator that starts from a fully hidden answer and fills in tokens over a handful of parallel steps. It can, in principle, decide what to write first instead of writing left to right. In practice, current multimodal diffusion reasoners decide this with a fixed, hand-written noise schedule. Every token is unhidden at the same rate regardless of whether it reads a fact off the image or derives a conclusion from other tokens. The fixed schedule therefore silently chooses a commitment order, and nobody has asked whether it is the right one.

We propose to learn that order. A small network, used only during training, assigns each answer token its own unmasking schedule conditioned on the image and question. The reverse model then learns to commit visually grounded propositions (objects, attributes, relations) early and to commit derived reasoning tokens late. It does this without any token labels at test time and at a cost of about ten sampling steps. In plain terms, the model learns to look before it concludes, and the fixed schedule is replaced by a trained one. A fine-grained hallucination taxonomy is the measuring instrument. We record the step at which every token was committed, so each wrong answer can be traced to the first wrong commitment and its error category. That turns the denoising timeline into an attribution axis that autoregressive models lack.

Core Claim

Our hypothesis is that cascading errors in parallel diffusion reasoners have two sources. One is wrong commitment order: derived tokens are fixed before the visual facts they depend on. The other is the few-step "representation gap": correlated tokens committed in the same step are sampled independently and contradict each other. A learned, input-conditioned forward process addresses both. Concretely, we parameterize the probability that token $i$ is still masked at diffusion time $t \in [0,1]$ as

$$\alpha_i(t \mid x) = 1 - F_{\mathrm{Kum}}\big(t;\, a_i(x), b_i(x)\big),$$

Here $x$ is the image, question and clean trace, and $F_{\mathrm{Kum}}$ is a Kumaraswamy mixture cumulative distribution function with token-specific shape parameters $a_i, b_i$ produced by the forward network. This form is monotone and pinned to the endpoints $0$ and $1$ by construction. We train on the standard masked-diffusion evidence lower bound plus a penalty on the variance of that bound across $t$. The penalty discourages schedules that pile all commitments into a few steps.

The prediction that separates this from simpler fixes has three parts. At ten steps, the learned schedule should beat both the hand-crafted schedules and a hand-written "perception first" schedule built from taxonomy labels. Its gains should concentrate in attribute and relation errors. And its advantage should be larger on perception-limited benchmarks than on reasoning-limited ones.

How This Sits in Prior Work

Two existing systems motivate the setting by showing that parallel diffusion avoids autoregressive error cascades. MMaDA-Parallel replaces think-then-draw autoregression with a discrete diffusion model in which text and image tokens co-evolve. It uses a fixed linear schedule for text and a fixed cosine schedule for images. DiffThinker treats a flow-matching denoising path as the reasoning path for spatial puzzles. Both keep the noising path fixed, and that fixed path is exactly what we learn.

On the text side, three papers constrain the design. Beyond Autoregression reweights the training loss toward hard tokens and decodes easy-first. It reaches 100% on Sudoku against 32.9% for a fine-tuned 13B LLaMA, which shows that order and difficulty matter. Its order, however, comes from a test-time confidence rule, not a learned forward process. WavefrontDiffusion likewise expands from confident tokens and admits it cannot recover when the model is confidently wrong. Confidently wrong is precisely the situation in visual hallucination. DiffTOD plans whole dialogues by diffusion to avoid drift, but needs 1024 denoising steps.

The method comes from the learnable-schedule line. Forward-Learned Discrete Diffusion learns input-dependent forward marginals with maximum coupling. It reaches perplexity 60.5 on ROCStories in 10 steps where fixed schedules produce incoherent text, and it adds no inference cost. The Minimum Variance Path principle shows that fixed paths leave a path-variance term out of the training objective and supplies the Kumaraswamy parameterization. Neither has been tested on multimodal reasoning. FLDD's REINFORCE-based gradients are a known source of instability, which is one reason we prefer the closed-form Kumaraswamy schedule over a fully free forward network.

The taxonomy and cascade literature supplies the measurements. Zhong et al. split multimodal hallucinations into existence, attribute, relation and imagination. They show that open-source models lose at least 31% accuracy when seeded with hallucinated history. MM-Snowball finds that visual evidence is suppressed rather than lost and builds dialogues from atomic propositions. Those propositions let us check individual committed tokens against ground truth.

The benchmarks disagree on where failure lives. MATH-Vision attributes 42.2% of errors to reasoning and 31.9% to vision. VCBench attributes about 62% of errors to perception. We treat this disagreement as a testable prediction, not a contradiction to resolve by argument. Finally, AgentHallu shows that locating the step responsible for an error is hard even for Gemini-2.5-Pro (41.1%) and only covers text. Our commitment-time attribution is a multimodal counterpart that comes from the generation process itself instead of from a second model's judgment.

Research Program

The base model is an openly available multimodal masked diffusion model in the MMaDA-Parallel family. We fine-tune it on visual question answering with written reasoning traces. Training traces come from sources disjoint from every evaluation set, for example template-generated chart rationales in the style of Synthesize Step-by-Step, so that VCBench, MATH-V and the snowball benchmarks remain held out. The forward network is a small transformer that reads the clean trace and image features and emits the per-token Kumaraswamy parameters. It is discarded after training. At inference, the reverse model needs a schedule without knowing the clean trace. Following FLDD, the reverse model is trained so that its own predicted masked set matches the learned marginals. As a fallback, a lightweight head predicts each position's schedule from the partial sequence; its cost is reported separately.

The analysis has three parts. First, we log for every token the step at which it was committed. We tag tokens by category using the MMHalSnowball and VCBench error taxonomies, applied automatically against MM-Snowball's atomic propositions and spot-checked by hand. With these tags we measure whether grounded tokens actually commit earlier under the learned schedule. Second, for each wrong answer we identify the earliest-committed incorrect token and its category. We then test whether fixing that single token by teacher-forcing the correct value and resampling the rest repairs the answer. That causal test is what makes the attribution meaningful. Third, we inject hallucinated history in the MMHalSnowball and MM-Snowball protocols and measure how often the committed answer flips to agree with the false history. A schedule that keeps visual-fact positions open until they are read off the image should resist such flips.

Baselines

The naive reference point is the same fine-tuned diffusion model with its native fixed schedule (linear for text), sampled at 10, 32 and 256 steps. We have found no reported result on Lacuna for MMaDA-Parallel on VCBench, MATH-V or MMHalSnowball; its only reported figure is a 6.9% alignment improvement over Bagel on ParaBench. We therefore produce these numbers ourselves under identical fine-tuning. The fixed-schedule model gets the same total training compute as the learned one, including the forward network's training FLOPs reallocated to extra denoiser steps, and the same tuning budget over schedule type. Beating it means higher accuracy at 10 steps and no loss at 256 steps.

The strongest inference-time ordering baseline is easy-first confidence decoding as in Beyond Autoregression, together with WavefrontDiffusion. Both run on the identical fixed-schedule checkpoint at matched step counts. These are the most direct competitors because they also change commitment order, at zero training cost. For reference, WavefrontDiffusion reports 82.03% on GSM8K with LLaDA-8B; that is a text number, so the comparison here is run on our benchmarks.

The key ablation is a hand-crafted taxonomy-ordered schedule. It uses the same per-token Kumaraswamy form but with parameters fixed by category: perception tokens early, derived tokens late, using the same category tags as our analysis. Comparing against it isolates whether learning the schedule matters or whether the right fixed order is enough. A second ablation removes the path-variance penalty, and a third removes input conditioning, giving one learned schedule per category.

The strongest autoregressive competitor on snowballing is Residual Visual Decoding, which blends full-context and vision-only predictions. On MMHalSnowball it raises LLaVA-1.5 from about 15% to 33% accuracy in the hallucinated-conversation setting (Zhong et al.). We also include CAVR from MM-Snowball; no numeric Snowballing Resistance Score is reported on Lacuna for CAVR, so we rerun it. Both are applied to an autoregressive model of matching parameter count, fine-tuned on the same traces. Beating RVD means exceeding its rerun accuracy on MMHalSnowball's hallucinated setting with comparable per-query latency.

For perception-heavy math, the reference numbers are 48.08% for Qwen2.5-VL-72B and 49.77% for Gemini 2.0 Flash on VCBench. On MATH-V, GPT-4o reports 30.39% and the best open model reported, InternVL-Chat-V1-2-Plus, reports 16.97%. These models are larger than ours. We use them as context and judge success against the matched-size baselines above.

Why not just…?

Why not just hard-code "perception first" using the taxonomy? This is the most serious threat, and it is the hand-crafted taxonomy-ordered ablation above. A fixed order needs category tags at inference, which the model does not have for a new question. It also cannot adapt per instance: which attribute is hard to perceive differs between images. And it does nothing about tokens committed together at ten steps contradicting each other, which FLDD's results tie to the forward process itself. If the hand-crafted order matches the learned one at ten steps, we will report that the benefit is ordering rather than learning, and the contribution shrinks to the attribution finding and a cheap schedule.

Why not just use confidence-based decoding (easy-first or wavefront)? It costs nothing and already changes order. But a hallucinating model is confident in the hallucination: near-100% flip rates on imagination errors (Zhong et al.) suggest confidence is a poor proxy for correctness, and WavefrontDiffusion names this failure. The hallucination-seeded flip-rate experiment tests it directly. If confidence ordering resists flips as well as the learned schedule, the training-time change is unnecessary.

Why not just patch an autoregressive model with RVD, CAVR or Visual Flow? These are real and cheap. Visual Flow cuts snowballing scores by 30–40% at 8–13% extra latency. However, absolute accuracy stays low: 33% with RVD, and below 2% in middle turns of MM-Snowball for LLaVA-1.5-7B before mitigation. They also cannot revise an earlier committed token. Our head-to-head on MMHalSnowball and MM-Snowball at matched size settles whether reformulating generation beats patching it. A loss there would mean the diffusion route is not worth its training cost for dialogue-style snowballing.

Why not just reweight hard tokens in the loss, as MGDM does? Reweighting changes how much the model learns each token, not when it commits it. The "no input conditioning" and "taxonomy-ordered" ablations, plus a MGDM-reweighted fixed-schedule run at matched compute, separate the two effects.

Proposed Experiments

All runs report accuracy at 10, 32 and 256 sampling steps on VCBench, MATH-V, MMHalSnowball and MM-Snowball, broken down by error category, with three seeds.

The first decisive check is the few-step regime. At ten steps, the learned schedule should beat the fixed schedule, confidence ordering and the taxonomy-ordered schedule, while the gap among them closes at 256 steps. That pattern is what we expect if few-step sampling is where order and joint consistency matter most.

The second check is the predicted asymmetry. Gains should be largest on VCBench, where perception dominates errors, and smallest on MATH-V, where reasoning errors dominate. The per-category breakdown should show attribute and relation errors falling most. If gains are uniform, the "look before concluding" story is wrong, even if accuracy rises.

The third check is mechanistic. Under the learned schedule, grounded tokens should commit measurably earlier than derived tokens. Seeded-hallucination flip rates should fall relative to the fixed schedule and to RVD.

The fourth check validates attribution. Teacher-forcing the earliest wrong committed token should repair a substantially larger share of failures than forcing a random wrong token. On MM-Snowball, where the injected perturbation is known, the attributed category should agree with the injected one. If neither holds, commitment time is not a useful attribution signal, and we will say so.