Introduction: From Vertical to Lateral Thinking
Traditional Large Language Models (LLMs) operate on an auto-regressive principle, predicting tokens sequentially from left to right. When tasked with complex reasoning, these models typically employ Chain-of-Thought (CoT), a method the authors describe as Vertical Thinking. While effective, Vertical Thinking is inherently linear and rigid; once a token is generated, it becomes a fixed part of the context, preventing the model from revising earlier steps in light of later realizations.
In contrast, human cognition often involves "lateral thinking"—a non-linear process where ideas are refined globally and thoughts can jump between different parts of a problem. This paper introduces Diffusion Chain of Lateral Thought (DCoLT), a framework that leverages Diffusion Language Models (DLMs) to enable this bidirectional, parallel refinement of reasoning trajectories.
Methodology: DCoLT and LLaDOU
DCoLT treats the entire reverse diffusion process as a reasoning trajectory. Unlike standard supervised fine-tuning that requires step-by-step labels, DCoLT utilizes outcome-based Reinforcement Learning (RL). Specifically, it employs Group Relative Policy Optimization (GRPO) to reward the model based on the correctness of the final output, allowing the model to discover its own optimal "thinking" path.
The framework is applied to two distinct diffusion architectures:
- SEDD (Continuous-time): The model's score function is optimized to derive a probabilistic policy.
- LLaDA (Discrete-time Masked Diffusion): The authors introduce the Unmask Policy Module (UPM). In masked diffusion, the model must decide which tokens to reveal at each step. By using a Plackett-Luce model, the UPM learns an optimal unmasking order, effectively identifying "anchor" tokens—critical numbers or operators—that should be resolved first to guide the rest of the generation. The resulting model is termed LLaDOU (LLaDA with Ordered Unmasking).
Key Results and Performance
The DCoLT framework demonstrates remarkable efficiency and performance gains across mathematical and coding benchmarks. Notably, LLaDOU (8B) achieved an accuracy of 88.1% on GSM8K and 44.6% on MATH, significantly outperforming baseline diffusion models and rivaling much larger auto-regressive counterparts.
Beyond standard benchmarks, the SEDD-based DCoLT model showed a profound ability to handle non-linear constraints, achieving 96.2% accuracy on 4x4 Sudoku. This far exceeds the 71.5% achieved by traditional CoT methods, which often struggle when a puzzle requires global consistency rather than sequential logic.
Insights and Data Efficiency
A core strength of DCoLT is its data efficiency. LLaDOU reached its high performance levels using only 15K training samples for RL—nearly two orders of magnitude less data than required by some leading auto-regressive models. This suggests that the non-causal, bidirectional nature of diffusion models is a more "natural" substrate for complex reasoning than forced linear sequences.
Furthermore, DCoLT enables Format-Free Reasoning. Because the model refines the entire string simultaneously, intermediate "thoughts" do not need to follow strict grammatical rules or linear progression. The model can "sketch" a global solution and fill in the technical details iteratively.
Limitations and Future Directions
While DCoLT represents a significant step forward, it currently relies on verifiable rewards (e.g., math or code where the answer is binary). Extending this to open-ended tasks like creative writing would require more sophisticated reward modeling. Additionally, the authors noted that performance scales with the number of diffusion steps and model size, suggesting that further hardware resources could unlock even greater reasoning capabilities.
Summary updated