The dominant paradigm in generative modeling—from Large Language Models (LLMs) to world models—relies on autoregressive decoding. By predicting the next token based on all previous ones, these models maintain high coherence but suffer from a fundamental "imagination bottleneck." Because each step depends on the completion of the last, inference latency scales linearly with the sequence length ($O(N)$), leaving modern parallel hardware significantly underutilized.
A growing body of research is breaking this bottleneck by shifting toward non-autoregressive (NAR) generation. Instead of a sequential chain, these models predict all components of a structured output—whether they are text tokens, image patches, or coordinates—simultaneously in a single parallel pass ($O(1)$).
The Autoregressive Bottleneck
In traditional models, generating a 1,000-token sequence requires 1,000 sequential forward passes. In time-critical applications like autonomous driving or real-time simulation, this delay is unacceptable. Lior Cohen and colleagues identify this as a primary limitation in Improving Token-Based World Models with Parallel Observation Prediction, where they introduce Parallel Observation Prediction (POP). By using dedicated placeholder tokens to predict an entire sequence of environment observations at once, they achieved a 15.4x speedup in "imagination" speed, allowing agents to simulate future trajectories far faster than real-time.
Parallelization via Diffusion and Denoising
One of the most effective ways to achieve parallel generation is through diffusion processes. Unlike autoregressive models that build a sequence from left to right, diffusion models start with a sequence of noise and refine all positions simultaneously.
Jason Yim and his team applied this to protein design in Fast non-autoregressive inverse folding with discrete diffusion, achieving a 23x speedup over sequential baselines. Similarly, in the realm of reasoning, Chenyang Shao explores Diffuse Thinking, where diffusion models act as "thought proposers." Instead of generating a single reasoning chain token-by-token, the model iteratively denoises multiple candidate thoughts in parallel, drastically reducing the cost of "test-time scaling" (giving a model more time to think).
Structured Queries and Spatial Parallelism
Beyond text, non-autoregressive methods are transforming how models handle structured spatial data. Traditional methods for tasks like human parsing or map construction often struggle with the "causal" nature of transformers, where the order of pixels or points is often arbitrary.
- Human-Centric Perception: Hong Chang developed RefHCM, which uses learnable queries to generate dense spatial outputs like parsing maps in a single step. This avoids the structural incoherence that can occur when a model predicts a mask one pixel at a time.
- Vectorized Mapping: In autonomous driving, Bencheng Liao introduced MapTR, which treats map elements as point sets. By predicting all coordinates for a geometric element simultaneously, the model runs 8x faster than sequential alternatives while maintaining better structural integrity.
- Vector Sketches: Hmrishav Bandyopadhyay proposed SketchINR, which represents sketches as continuous functions. This allows for parallel coordinate decoding, resulting in 100x faster reconstruction compared to recurrent models.
Solving the Multi-Solution Problem
A major challenge in non-autoregressive learning is the "one-of-many" problem. In combinatorial optimization or puzzles like Sudoku, there may be multiple valid solutions. If a parallel model is trained to minimize the distance to all valid solutions at once, it often predicts a blurry average of them that satisfies none.
Ruizhong Qiu addressed this in DIMES by parameterizing solution distributions in a continuous space, allowing for parallel sampling of candidate solutions. This approach bypasses the linear time complexity of building solutions node-by-node, enabling neural solvers to scale to problems with tens of thousands of variables. Similarly, Yatin Nandwani proposed the SELECTR framework, which uses reinforcement learning to help non-autoregressive models pick a single target solution to follow, preventing the "averaging" effect.
Hybrid Strategies: Speculative Decoding
For tasks where pure non-autoregressive generation still lags in quality, researchers are using NAR models as "drafters" for autoregressive models. Rui Pan introduced FailFast, a speculative decoding framework that uses a diffusion LLM to predict up to 70 tokens at once. The main autoregressive model then verifies these tokens in parallel. This "draft-then-verify" approach provides a 4.9x speedup while maintaining the exact output distribution of the larger, slower model.
By removing the requirement that the future must be generated one step at a time, these parallel architectures allow generative AI to move from slow, deliberative processes to the high-speed, reactive performance required for real-world interaction.