Standard language models generate text like a typewriter: one token at a time, from left to right. While this "autoregressive" approach ensures high quality, it is inherently slow because each new word must wait for the previous one to finish. Non-autoregressive (NAR) models aim to break this bottleneck by generating all tokens in a sequence simultaneously, offering speedups of 10x to 20x.
However, generating a sequence in one shot introduces a fundamental challenge: the multimodality problem. Because tokens are predicted independently, the model lacks a mechanism to coordinate between positions. This often results in "stuttering"—where the model predicts the same high-probability word in multiple adjacent slots—or "output collapse," where the sequence is globally incoherent despite being locally plausible.
The Coordination Challenge and Iterative Refinement
The primary hurdle for NAR models is that they cannot "see" what they are currently writing in other parts of the sentence. In Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement, Jason Lee and Kyunghyun Cho address this by treating generation as a denoising process. Instead of a single pass, the model generates a rough draft and then iteratively polishes it, allowing information to flow between tokens over several steps.
This concept has evolved into Diffusion Language Models (DLMs), which use many small refinement steps to transform noise into coherent text. To make these models practical, researchers like Xiaojing Qi and Lun Du have introduced Hierarchy Decoding, a "divide-and-conquer" strategy that recursively partitions the sequence to maintain global coherence without sacrificing the speed of parallel execution.
Solving the Stuttering Problem
The tendency for NAR models to repeat tokens (e.g., "the the the") stems from traditional training objectives that penalize a model if it predicts the right word in the wrong position. Cunxiao Du and Zhaopeng Tu proposed Order-Agnostic Cross Entropy (OAXE) in Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation. By allowing the model to predict the correct set of words without being strictly tied to specific positions during training, OAXE effectively reduces the "multimodality" that causes repetitive outputs.
In specialized domains like antibody design, these repetitions aren't just a nuisance—they are biologically dangerous. Minsu Kim and Jinkyoo Park developed the ASSD framework, which decouples sequence generation from structure prediction. They use a composition-based objective to ensure that the parallel-generated amino acid sequences are diverse and functional, preventing the "stuttering" that leads to protein aggregation.
Accelerating Inference through Confidence and Caching
Recent work focuses on making parallel generation as efficient as possible. Standard diffusion models are often slowed by "playing it safe," only unmasking the most certain tokens. The Explore-Then-Exploit (ETE) strategy introduced by Hengyu Fu in From Bits to Rounds intentionally unmasks high-uncertainty tokens to trigger "confidence cascades," significantly reducing the number of rounds needed to reach a final output.
Furthermore, FlashDLM by Zhanqiu Hu and Zhiru Zhang combines efficient KV caching with lightweight autoregressive supervision. This hybrid approach provides the local "guidance" necessary to keep parallel generation on track, ensuring that the speed of NAR models finally comes with the reliability of traditional sequential ones.