Abstract
Diverse outputs in text generation are necessary for effective exploration in complex reasoning tasks, such as code generation and mathematical problem solving. Such Pass@$k$ problems benefit from distinct candidates covering the solution space. However, traditional sampling approaches often waste computational resources on repetitive failure modes. While Diffusion Language Models have emerged as a competitive alternative to the prevailing Autoregressive paradigm, they remain susceptible to this redundancy, with independent samples frequently collapsing into similar modes. To address this, we propose a training free, low cost intervention to enhance generative diversity in Diffusion Language Models. Our approach, Orthogonal Diverse Diffusion (ODD), modifies intermediate samples in a batch sequentially, where each sample is repelled from the feature space of previous samples, actively penalising redundancy. Unlike prior methods that require retraining or beam search, our strategy incurs negligible computational overhead, while ensuring that each sample contributes a unique perspective to the batch. We evaluate our method on the HumanEval and GSM8K benchmarks using the LLaDA-8B-Instruct model. Our results demonstrate significantly improved diversity and Pass@$k$ performance across various temperature settings.
Introduction: The Redundancy Problem in Sampling
When solving complex reasoning tasks like code generation or mathematical problem solving, a single attempt is often insufficient. Instead, practitioners typically use the Pass@$k$ metric: generating $k$ independent samples and checking if at least one is correct. The effectiveness of this strategy depends heavily on the diversity of the generated samples.
Standard sampling techniques, such as increasing the temperature, often fail to produce meaningful diversity. Instead, models frequently suffer from "mode collapse," where multiple samples fall into the same repetitive failure modes. This redundancy wastes computational resources without increasing the probability of finding a correct solution.
Methods: Orthogonal Diverse Diffusion (ODD)
To mitigate redundancy, we introduce Orthogonal Diverse Diffusion (ODD), a training-free inference-time intervention. ODD ensures that each sample in a batch explores a distinct region of the semantic space by sequentially repelling new samples from the feature space of previously generated ones.
The process consists of two primary steps:
- Feature Extraction: For each sample $\mathbf{x}_i$ in a batch, we extract a feature vector $v_i$ that represents its semantic profile. This is typically achieved by pooling the sequence logits.
- Sequential Repulsion: We process the batch sequentially. For the $i$-th sample, we calculate a diversity loss $\mathcal{L}_{orth}$ based on its distance from the orthogonal basis $B_{<i}$ formed by the previous $i-1$ samples. We then update the sample's logits to push it away from this occupied subspace.
To maintain generation quality, the repulsion term is modulated by a quality-aware weighting and an annealing schedule. The diversity push is strongest during the early stages of the diffusion process, where high-level semantic structures are formed, and tapers off as the model refines low-level details.
Experimental Setup
We evaluated ODD using the LLaDA-8B-Instruct model, a state-of-the-art Diffusion Language Model. Our experiments focused on two challenging reasoning benchmarks:
- HumanEval: 164 Python programming problems designed to test functional correctness.
- GSM8K: A benchmark of 8,500 grade-school math word problems requiring multi-step reasoning.
Key Results and Insights
Our results demonstrate that ODD provides a significant performance boost with minimal computational overhead (adding only 4-6% to wall-clock time).
- Improved Pass@$k$: On HumanEval, ODD improved the Pass@16 score from 42.7% to 51.3% at a temperature of 1.0. Similarly, on GSM8K, the Pass@16 score increased from 81.3% to 87.9%.
- The Diffusion Advantage: Diffusion models refine the entire sequence simultaneously, providing a "global view" that makes them uniquely suited for mid-generation steering compared to Autoregressive models.
- Coherence at High Temperatures: At high temperatures where standard sampling often becomes incoherent, ODD acts as a regularizer, guiding the model toward high-confidence regions while maintaining diversity.
Conclusion and Future Work
ODD offers a "free lunch" for Pass@$k$ problems by significantly enhancing diversity in Diffusion Language Models without the need for retraining. While the current implementation uses simple logit-based features, future work could explore more sophisticated semantic encoders or incorporate positional information to further refine the diversity term. Additionally, adapting the core logic of orthogonal repulsion for Autoregressive models remains an open and promising research direction.
Summary updated