In machine learning, we often use models to train other models. Whether through back-translation in machine translation, question generation for QA, or captioning for vision-language tasks, synthetic data has become a primary fuel for performance gains. However, a recurring discovery across research domains is that the "best" outputs from a generative model—those with the highest likelihood—are often the worst for training.
Instead of using deterministic methods like beam search to find the most probable sequence, researchers have found that injecting randomness through stochastic sampling creates a significantly more effective training signal.
The Failure of Maximum Likelihood
When we generate synthetic data using deterministic methods like beam search, the model acts as a "mode-seeker." It consistently picks the most frequent, safe, and predictable patterns. While this results in grammatically correct and high-quality text, it lacks the lexical and structural variety of natural language.
Sergey Edunov and colleagues demonstrated this at scale in Understanding Back-Translation at Scale. They found that in high-resource machine translation, synthetic data generated via sampling or noised beam search outperformed standard beam search. The reason is intuitive: deterministic outputs are "too easy." They are overly regularized and predictable, providing little information for a student model to learn from. In contrast, sampled data is "noisier" and more complex, forcing the model to learn more robust representations.
This phenomenon is not limited to translation. In Question Answering, Md. Arafat Sultan showed in On the Importance of Diversity in Question Generation for QA that using diversity-promoting nucleus sampling boosts downstream performance even when standard metrics like BLEU suggest the generated questions are of "lower quality." The "boring" but accurate questions produced by beam search simply don't cover the linguistic and factual variety a model needs to generalize.
Diversity as a Training Signal
Stochastic sampling—specifically techniques like top-k sampling or nucleus (top-p) sampling—allows a model to explore the "long tail" of a distribution. This exploration serves several critical functions:
- Lexical Enrichment: Aarón Galiano Jiménez found that stochastic decoding in knowledge distillation acts as a "lexical enricher," exposing student models to a broader range of valid but less frequent tokens (Beyond the Mode).
- Bias Mitigation: Chenyang Li noted that random sampling prevents models from amplifying their own biases during self-learning, acting as a natural regularizer (Improving Zero-shot Multilingual Neural Machine Translation for Low-Resource Languages).
- Robustness to Noise: By introducing structural diversity and irregularity, sampling prevents models from overfitting to the "perfect" patterns of machine-generated text. This is particularly vital in speech translation (DUB) and grammatical error correction (Aomi Koyama, Comparison of Grammatical Error Correction Using Back-Translation Models).
Scaling Beyond Text
The principle that "stochasticity breeds robustness" extends into multimodal and physical domains. In vision-language pre-training, Junnan Li used nucleus sampling in the BLIP framework to generate diverse synthetic captions, providing a richer learning signal for image-text matching. Similarly, Yue Zhao applied this to video narrations in LaViLa, finding that the resulting diversity improved the generalization of contrastive encoders.
Even in robotics, Abraham George demonstrated that the inherent stochasticity of LLM outputs can be used to generate diverse successful trajectories from a single human demonstration, creating a broader data distribution that makes robotic policies more resilient to environmental changes (LLM Trainer).
Navigating the Trade-off
The challenge of using stochastic sampling is the trade-off between diversity and faithfulness. If the sampling is too aggressive, the synthetic data becomes nonsensical or loses its label consistency.
Danilo Croce and colleagues explored this boundary in Learning to Generate Examples for Semantic Processing Tasks, using nucleus sampling to navigate the space between repetitive, generic examples and those that drift too far from the target semantic class. The goal is to find the "sweet spot" of noise—enough to challenge the model, but not so much that it learns incorrect associations.
By moving "beyond the mode," researchers like Bei Li and the NiuTrans team have shown that the path to state-of-the-art performance isn't through cleaner data, but through more diverse, stochastically-generated complexity.