Large Margin Neural Language Model

Jiaji Huang, Yi Li, Wei Ping, Liang Huang

Traditionally, the gold standard for training language models (LMs) has been the minimization of perplexity (PPL). However, research presented at EMNLP 2018 by Baidu and Oregon State University challenges this convention. They argue that while low perplexity indicates a model can predict the next word well, it does not necessarily translate to better performance in downstream tasks like Automatic Speech Recognition (ASR) or Machine Translation (MT).

The Discriminative Gap

LMs in ASR and MT systems are typically used to re-score a list of candidate hypotheses. A standard LM trained on a massive corpus of "good" text learns to assign high probabilities to common sequences. The authors identify a "discriminative gap": a model might assign a higher score to an error-filled hypothesis than to the ground-truth sentence simply because the error contains high-frequency word sequences. For instance, an ASR system might output "a decade as concerns" instead of "its defeat is confirmed." A standard LM might prefer the former because "a decade as" is a common phrase, even though it is contextually incorrect.

Methodology: From Generative to Discriminative

The researchers proposed shifting the training objective from generative (minimizing PPL) to discriminative (maximizing a margin). They explored several formulations to achieve this:

  1. The Naive Approach: Initially, they attempted to maximize the score of the reference sentence while minimizing the score of the hypothesis. This led to extreme training instability, as the gradients from "bad" sentences dominated the process.
  2. LMLM (Large Margin Language Model): To solve the instability, they introduced a hinge loss objective. This ensures the log-likelihood of the ground-truth sentence ($x_i$) is at least a margin ($\tau$) higher than the log-likelihood of an incorrect hypothesis ($x_{i,j}$). If the gap is already sufficient, the loss is zero.
  3. rLMLM (Ranking-based LMLM): A more advanced version that enforces margins across an entire list of candidates, ranking them based on task-specific metrics like Word Error Rate (WER) or BLEU scores.
Figure 3: Training loss comparison. (a) illustrates the instability and extreme values encountered with the naive discriminative objective, whereas (b) shows the stable convergence achieved using the LMLM hinge loss.

The Perplexity Paradox

A key finding of the study is what could be called the "Perplexity Paradox." When evaluated, LMLM and rLMLM exhibited significantly higher (worse) perplexity than standard RNNLMs. Despite this, they performed substantially better on actual tasks. This confirms that PPL is often a poor proxy for the utility of an LM in re-scoring applications.

By training on the actual mistakes generated by ASR or MT systems, the LMLM learns to specifically penalize the types of errors those systems are prone to making. Furthermore, the large margin objective naturally corrects for length bias—a common issue where standard LMs prefer shorter sentences because they involve fewer negative log-probability additions.

Figure 6: Distribution of the log-probability gap between ground truth and hypotheses. (a) shows the failure of the naive approach, while (b) demonstrates that LMLM (green) successfully shifts the distribution to the right compared to a standard RNNLM (blue), indicating a larger, more consistent margin.

Key Results and Impact

The large margin approach demonstrated consistent improvements across multiple benchmarks:

  • ASR (Wall Street Journal): Reduced WER from 7.69% to 5.48%.
  • ASR (10K Hour Noisy Dataset): Reduced WER from 20.90% to 19.49%, proving robustness in noisy, real-world conditions.
  • Machine Translation: Increased the BLEU score on the NIST 08 test set from 31.52 to 33.13.

While the ranking-based approach (rLMLM) is more computationally expensive due to the need for pairwise comparisons, the results suggest that discriminative training is a powerful tool for bridging the gap between general linguistic modeling and task-specific performance. Future work remains to be done on scaling these pairwise comparisons to handle larger candidate sets more efficiently.

Summary updated

Abstract

We propose a large margin criterion for training neural language models. Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences. However, we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation. The proposed method aims to enlarge the margin between the "good" and "bad" sentences in a task-specific sense. It is trained end-to-end and can be widely applied to tasks that involve re-scoring of generated text. Compared with minimum-PPL training, our method gains up to 1.1 WER reduction for speech recognition and 1.0 BLEU increase for machine translation.
BibTeX
@inproceedings{huang2018large,
  author = {Jiaji Huang and Yi Li and Wei Ping and Liang Huang},
  title = {{Large Margin Neural Language Model}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  url = {https://openreview.net/forum?id=Bk4yjGGO-r},
}
Figure