Minimum Risk Training for Sequence Generation

This direction is split into other directions: minimum risk training. Its last version stays readable and citable here.

Version 1 of 162 papers77 conceptsfrozen 2026-10-04

In sequence-to-sequence tasks like machine translation or speech recognition, there is a fundamental disconnect between how models are trained and how they are evaluated. We typically train models using Maximum Likelihood Estimation (MLE), which focuses on predicting the next correct token in a sequence. However, we evaluate these same models using global, sequence-level metrics like BLEU, ROUGE, or Word Error Rate (WER).

This discrepancy creates a "performance plateau." A model might become excellent at predicting the next word in a vacuum but fail to produce a coherent, high-quality sentence because its training objective never actually "saw" the final evaluation metric. Minimum Risk Training (MRT)—also known as Expected Risk Minimization—bridges this gap by allowing models to optimize directly for the metrics we care about.

The Misalignment Problem

Standard training relies on "teacher forcing," where the model is given the ground-truth prefix at every step. This leads to two major issues:

  1. Exposure Bias: During inference, the model must generate tokens based on its own previous (and potentially erroneous) predictions, a situation it never encountered during training. Chaojun Wang and Rico Sennrich have shown that this bias is a primary driver of "hallucinations," where models generate fluent but factually incorrect text when faced with unfamiliar data On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation.
  2. Metric Mismatch: MLE treats all "wrong" tokens equally. If the target is "cat," predicting "kitten" is penalized just as heavily as predicting "refrigerator," even though "kitten" would result in a much better BLEU score.

Optimizing the Expectation

Because metrics like BLEU are non-differentiable—you cannot take a gradient through a discrete count of matching n-grams—we cannot use them directly in standard backpropagation.

MRT solves this by treating the evaluation metric as a cost function and minimizing the expected risk (the average loss) across a distribution of possible candidate sequences. Instead of asking "What is the probability of the single correct answer?", the model asks: "If I sample several possible translations, what is their average quality, and how can I shift my parameters to make the high-quality ones more likely?"

As Shiqi Shen, Yong Cheng, and Maosong Sun demonstrated, this approach allows the model to explore the space of its own generated outputs during training, effectively "rehearsing" for the conditions it will face during inference Minimum Risk Training for Neural Machine Translation.

From Translation to Speech and Style

The intuition behind MRT has proven robust across diverse domains:

Why It Works: The Regularization Effect

Beyond just aligning metrics, MRT acts as a powerful regularizer. Liang Lu and Jinyu Li noted that sequence-level risk minimization makes models significantly more robust to decoding hyperparameters On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer. Because the model is trained on a distribution of its own N-best hypotheses, it becomes less sensitive to the specific beam size or length normalization used during final deployment.

By shifting the training objective from "predict the next token" to "maximize the quality of the whole sequence," MRT ensures that the model's internal representations are tuned for the actual task at hand, rather than a statistical proxy.

Papers in this version

Showing 60 of 62 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_minimum_risk_training_for_sequence_gener_e5cac78c,
  title = {Minimum Risk Training for Sequence Generation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/minimum-risk-training-for-sequence-generation/rdr_e5cac78cb4ab400499a70efae7f1041e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_minimum_risk_training_for_sequence_gener_e5cac78c_v1,
  title = {Minimum Risk Training for Sequence Generation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/minimum-risk-training-for-sequence-generation/rdr_e5cac78cb4ab400499a70efae7f1041e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}