Abstract
Modern text generation faces a persistent "quality-diversity tradeoff." While Auto-Regressive (AR) models excel at local fluency, they often struggle with global diversity and fine-grained controllability. This paper introduces Diffusion-EAGS, a novel framework that reinterprets Masked Language Modeling (MLM) through the lens of a conditional Markov Random Field (cMRF). By utilizing entropy—a measure of model uncertainty—to guide both training and inference, Diffusion-EAGS achieves state-of-the-art performance in diversity and controllability while maintaining high generation quality in as few as five diffusion steps.
Introduction
Most contemporary Large Language Models (LLMs) rely on Auto-Regressive (AR) generation, predicting tokens sequentially from left to right. This approach has two primary drawbacks: a lack of output diversity, as models often converge on "safe" or repetitive sequences, and poor controllability, as the sequential nature makes it difficult to satisfy constraints that appear later in a sentence without compromising the beginning.
Non-Auto-Regressive (Non-AR) models, such as Discrete Diffusion and Masked Language Models (MLMs), offer a potential solution by refining entire sequences simultaneously. However, as shown in Figure 1, these approaches have their own limitations: MLMs often struggle with multi-step refinement, while standard Diffusion models may lack the deep contextual background knowledge inherent in pre-trained MLMs.
The Diffusion-EAGS Framework
Diffusion-EAGS bridges the gap between these paradigms by treating text generation as an energy minimization problem within a cMRF framework. The core innovation lies in using entropy as a functional tool to guide the model's focus during both training and generation.
Entropy-based Noise Scheduling (ENS)
During the training (forward) process, the model employs ENS. Unlike random masking, ENS progressively masks tokens based on their entropy in ascending order. By masking the "easiest" or most certain tokens first, the model learns a structured denoising path. This ensures that the training objective aligns closely with the iterative refinement process used during inference.
Entropy-Adaptive Gibbs Sampling (EAGS)
For inference (the backward process), the model utilizes EAGS. At each timestep, the model calculates the entropy for every position in the sequence and prioritizes updating the most uncertain (high-entropy) tokens. This allows the model to establish a stable syntactic foundation before filling in specific semantic details, effectively preventing the "semantic drift" common in other non-AR models.
Experimental Evaluation
The researchers evaluated Diffusion-EAGS across a diverse array of tasks, including narrative generation (RocStories), text detoxification (Paradetox), and ethical reasoning (Deontology).
Key Results
- Diversity-Quality Tradeoff: Diffusion-EAGS consistently outperformed baselines like GPT-2 and DiffusionBERT. It achieved lower Perplexity (PPL) while significantly increasing Vendi Scores, a metric for output diversity.
- Controllability: In keyword-constrained tasks, the model demonstrated a superior ability to integrate specific terms into coherent narratives compared to AR models, which often produced awkward phrasing when forced to meet similar constraints.
- Efficiency: The model is remarkably efficient, reaching peak performance in just 5 diffusion steps, making it competitive with AR models in terms of generation speed.
- Human Preference: In "GPT-as-a-Judge" evaluations, Diffusion-EAGS was preferred over other diffusion baselines more than 65% of the time for question generation tasks.
Discussion and Conclusion
Diffusion-EAGS demonstrates that entropy is not merely an evaluation metric but a powerful compass for guiding generative models. By focusing on high-uncertainty regions, the model guarantees a reduction in sequence energy, leading to more coherent and diverse outputs.
While currently focused on generation tasks using a RoBERTa-base backbone, future work may involve scaling the framework to larger encoder-decoder architectures like T5. Additionally, exploring the framework's utility in discriminative tasks remains a promising avenue for research. Ultimately, Diffusion-EAGS provides a theoretically grounded path toward more controllable and diverse language generation.
Summary updated