Abstract
Despite some empirical success at correcting exposure bias in machine translation, scheduled sampling algorithms suffer from a major drawback: they incorrectly assume that words in the reference translations and in sampled sequences are aligned at each time step. Our new differentiable sampling algorithm addresses this issue by optimizing the probability that the reference can be aligned with the sampled output, based on a soft alignment predicted by the model itself. As a result, the output distribution at each time step is evaluated with respect to the whole predicted sequence. Experiments on IWSLT translation tasks show that our approach improves BLEU compared to maximum likelihood and scheduled sampling baselines. In addition, our approach is simpler to train with no need for sampling schedule and yields models that achieve larger improvements with smaller beam sizes.
Introduction
Neural Machine Translation (NMT) models are typically trained using "Teacher Forcing," where the ground-truth reference word is provided as the prefix for predicting the next token. While effective, this creates a discrepancy between training and inference, known as exposure bias. At test time, the model relies on its own previous (and potentially erroneous) predictions, leading to compounding errors.
The Alignment Problem in Scheduled Sampling
Scheduled Sampling (SS) was proposed to mitigate exposure bias by occasionally replacing ground-truth tokens with model-generated samples during training. However, SS assumes a rigid temporal alignment: the $t$-th word of a sampled sequence must match the $t$-th word of the reference. In translation, where word order is often flexible, a model might generate a correct word at the "wrong" time step. Standard SS would penalize such valid variations, providing a noisy or incorrect training signal.
Soft Aligned Maximum Likelihood (SAML)
To address this, researchers at the University of Maryland introduced Soft Aligned Maximum Likelihood (SAML). SAML decouples the loss from specific time indices by using a soft-alignment mechanism.
SAML incorporates two main technical innovations:
- Differentiable Sampling: Using the Straight-Through (ST) Gumbel-Softmax estimator, the model can sample discrete words in the forward pass while maintaining a differentiable path for backpropagation.
- Flexible Reference Alignment: Instead of a 1-to-1 match, SAML calculates the probability that a reference word can be aligned anywhere in the sampled sequence. This alignment is "soft," based on the similarity between the model's hidden states and reference word embeddings.
Experimental Evaluation
The approach was tested on three IWSLT benchmarks: German-English (de-en), English-German (en-de), and Vietnamese-English (vi-en). These datasets provide a range of resource levels and word-order complexities.
Key Results
- Performance Gains: SAML consistently outperformed Maximum Likelihood (MLE) and Scheduled Sampling baselines, with BLEU improvements ranging from +0.5 to +1.0.
- Training Stability: Unlike standard SS, which requires a complex annealing schedule to avoid training collapse, SAML is robust enough to work with a fixed sampling rate, making it significantly easier to tune.
- Search Efficiency: SAML showed significant gains at small beam sizes and even with greedy search, suggesting the model learns to be more self-correcting and less dependent on expensive search algorithms to find a good path.
Conclusion
SAML offers a simpler and more effective way to handle exposure bias in NMT. By allowing for flexible word order during training, it aligns more closely with the actual requirements of translation tasks, leading to more robust models that perform well even with limited search budgets.
Summary updated