SetCommit: Coordinated Token Commitments for Few-Step Diffusion Language Model Decoding

Yuannuo Feng, Yubing Ye, Wenshuai Yao, Yizhe Chen, Ngai Wong, Wenyong Zhou, Wang KangOpenReview 2027 · Submission to ICLR 2027 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Diffusion language models generate multiple token positions in parallel, but few-step decoding makes parallel commitment difficult: tokens that appear reliable individually may be jointly inconsistent, while premature commitments can prevent later context-dependent refinement. In this paper, we introduce \textsc{SetCommit}, a training-free method for coordinated parallel token commitment. Instead of selecting tokens independently by confidence, \textsc{SetCommit} selects a commitment set that balances token reliability against local joint commitment risk, preserving conditioning opportunities for subsequent denoising steps. It further performs budget-neutral local exchanges, replacing vulnerable earlier commitments when newly revealed context supports more reliable alternatives. \textsc{SetCommit} requires no retraining or architectural modification and is compatible with existing diffusion decoding strategies. Experiments with LLaDA-8B and Dream-7B on GSM8K, MATH-500, MBPP, and HumanEval show accuracy/pass@1 gains of up to 7.4 percentage points over native decoding at 32 model evaluations. At common quality targets, geometric-mean incremental speedups range from $1.44$ to $1.74\times$. Ablations support the benefits of coordinated selection and bounded remasking. Code is available at \url{https://anonymous.4open.science/r/review-artifact-48dc4648-8B86}.
BibTeX
@misc{feng2027setcommit,
  author = {Yuannuo Feng and Yubing Ye and Wenshuai Yao and Yizhe Chen and Ngai Wong and Wenyong Zhou and Wang Kang},
  title = {{SetCommit: Coordinated Token Commitments for Few-Step Diffusion Language Model Decoding}},
  year = {2027},
  url = {https://openreview.net/forum?id=LxhKURNgpg},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2027},
}

Loading related content...

Figure