In sequence-to-sequence tasks like machine translation or speech recognition, there is a fundamental disconnect between how models are trained and how they are evaluated. We typically train models using Maximum Likelihood Estimation (MLE), which focuses on predicting the next correct token in a sequence. However, we evaluate these same models using global, sequence-level metrics like BLEU, ROUGE, or Word Error Rate (WER).
This discrepancy creates a "performance plateau." A model might become excellent at predicting the next word in a vacuum but fail to produce a coherent, high-quality sentence because its training objective never actually "saw" the final evaluation metric. Minimum Risk Training (MRT)—also known as Expected Risk Minimization—bridges this gap by allowing models to optimize directly for the metrics we care about.
The Misalignment Problem
Standard training relies on "teacher forcing," where the model is given the ground-truth prefix at every step. This leads to two major issues:
- Exposure Bias: During inference, the model must generate tokens based on its own previous (and potentially erroneous) predictions, a situation it never encountered during training. Chaojun Wang and Rico Sennrich have shown that this bias is a primary driver of "hallucinations," where models generate fluent but factually incorrect text when faced with unfamiliar data On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation.
- Metric Mismatch: MLE treats all "wrong" tokens equally. If the target is "cat," predicting "kitten" is penalized just as heavily as predicting "refrigerator," even though "kitten" would result in a much better BLEU score.
Optimizing the Expectation
Because metrics like BLEU are non-differentiable—you cannot take a gradient through a discrete count of matching n-grams—we cannot use them directly in standard backpropagation.
MRT solves this by treating the evaluation metric as a cost function and minimizing the expected risk (the average loss) across a distribution of possible candidate sequences. Instead of asking "What is the probability of the single correct answer?", the model asks: "If I sample several possible translations, what is their average quality, and how can I shift my parameters to make the high-quality ones more likely?"
As Shiqi Shen, Yong Cheng, and Maosong Sun demonstrated, this approach allows the model to explore the space of its own generated outputs during training, effectively "rehearsing" for the conditions it will face during inference Minimum Risk Training for Neural Machine Translation.
From Translation to Speech and Style
The intuition behind MRT has proven robust across diverse domains:
- Speech Recognition: In end-to-end speech systems, Chao Weng and colleagues applied Minimum Bayes Risk (MBR) training to RNN-Transducers. By minimizing the expected edit distance (the basis of WER) rather than just maximizing likelihood, they achieved significant accuracy gains on massive datasets Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition.
- Text Style Transfer: Yevgeniy Puzikov and Iryna Gurevych found that MRT provides a more stable training signal than adversarial methods, avoiding common issues like mode collapse while directly optimizing for style accuracy Empirical Evaluation of Supervision Signals for Style Transfer Models.
- Document-Level Context: Danielle Saunders and Bill Byrne extended MRT to optimize for document-level metrics, allowing models to maintain consistency across multiple sentences by aggregating risk over entire batches Using Context in Neural Machine Translation Training Objectives.
Why It Works: The Regularization Effect
Beyond just aligning metrics, MRT acts as a powerful regularizer. Liang Lu and Jinyu Li noted that sequence-level risk minimization makes models significantly more robust to decoding hyperparameters On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer. Because the model is trained on a distribution of its own N-best hypotheses, it becomes less sensitive to the specific beam size or length normalization used during final deployment.
By shifting the training objective from "predict the next token" to "maximize the quality of the whole sequence," MRT ensures that the model's internal representations are tuned for the actual task at hand, rather than a statistical proxy.