Traditionally, the gold standard for training language models (LMs) has been the minimization of perplexity (PPL). However, research presented at EMNLP 2018 by Baidu and Oregon State University challenges this convention. They argue that while low perplexity indicates a model can predict the next word well, it does not necessarily translate to better performance in downstream tasks like Automatic Speech Recognition (ASR) or Machine Translation (MT).
The Discriminative Gap
LMs in ASR and MT systems are typically used to re-score a list of candidate hypotheses. A standard LM trained on a massive corpus of "good" text learns to assign high probabilities to common sequences. The authors identify a "discriminative gap": a model might assign a higher score to an error-filled hypothesis than to the ground-truth sentence simply because the error contains high-frequency word sequences. For instance, an ASR system might output "a decade as concerns" instead of "its defeat is confirmed." A standard LM might prefer the former because "a decade as" is a common phrase, even though it is contextually incorrect.
Methodology: From Generative to Discriminative
The researchers proposed shifting the training objective from generative (minimizing PPL) to discriminative (maximizing a margin). They explored several formulations to achieve this:
- The Naive Approach: Initially, they attempted to maximize the score of the reference sentence while minimizing the score of the hypothesis. This led to extreme training instability, as the gradients from "bad" sentences dominated the process.
- LMLM (Large Margin Language Model): To solve the instability, they introduced a hinge loss objective. This ensures the log-likelihood of the ground-truth sentence ($x_i$) is at least a margin ($\tau$) higher than the log-likelihood of an incorrect hypothesis ($x_{i,j}$). If the gap is already sufficient, the loss is zero.
- rLMLM (Ranking-based LMLM): A more advanced version that enforces margins across an entire list of candidates, ranking them based on task-specific metrics like Word Error Rate (WER) or BLEU scores.
The Perplexity Paradox
A key finding of the study is what could be called the "Perplexity Paradox." When evaluated, LMLM and rLMLM exhibited significantly higher (worse) perplexity than standard RNNLMs. Despite this, they performed substantially better on actual tasks. This confirms that PPL is often a poor proxy for the utility of an LM in re-scoring applications.
By training on the actual mistakes generated by ASR or MT systems, the LMLM learns to specifically penalize the types of errors those systems are prone to making. Furthermore, the large margin objective naturally corrects for length bias—a common issue where standard LMs prefer shorter sentences because they involve fewer negative log-probability additions.
Key Results and Impact
The large margin approach demonstrated consistent improvements across multiple benchmarks:
- ASR (Wall Street Journal): Reduced WER from 7.69% to 5.48%.
- ASR (10K Hour Noisy Dataset): Reduced WER from 20.90% to 19.49%, proving robustness in noisy, real-world conditions.
- Machine Translation: Increased the BLEU score on the NIST 08 test set from 31.52 to 33.13.
While the ranking-based approach (rLMLM) is more computationally expensive due to the need for pairwise comparisons, the results suggest that discriminative training is a powerful tool for bridging the gap between general linguistic modeling and task-specific performance. Future work remains to be done on scaling these pairwise comparisons to handle larger candidate sets more efficiently.
Summary updated