Regularized Minimum Error Rate Training

Michel Galley, Chris Quirk, Colin Cherry, Kristina ToutanovaEMNLP 2013 · OpenReview

Abstract

Minimum Error Rate Training (MERT) remains one of the preferred methods for tuning linear parameters in machine translation systems, yet it faces significant issues. First, MERT is an unregularized learner and is therefore prone to overfitting. Second, it is commonly used on a noisy, non-convex loss function that becomes more difficult to optimize as the number of parameters increases. To address these issues, we study the addition of a regularization term to the MERT objective function. Since standard regularizers such as $L_2$ are inapplicable to MERT due to the scale invariance of its objective function, we turn to two regularizers—$L_0$ and a modification of $L_2$—and present methods for efficiently integrating them during search. To improve search in large parameter spaces, we also present a new direction finding algorithm that uses the gradient of expected BLEU to orient MERT’s exact line searches. Experiments with up to 3,600 features show that these extensions of MERT yield results comparable to PRO, a learner often used with large feature sets.

Introduction

In Statistical Machine Translation (SMT), "tuning" is the process of finding optimal weights for various features to maximize translation quality. For years, Minimum Error Rate Training (MERT) was the standard approach because it directly optimizes evaluation metrics like BLEU. However, as MT models have grown in complexity, MERT has struggled with two major flaws: overfitting due to a lack of regularization and search failure in high-dimensional spaces.

MERT's loss function is piecewise constant, resembling a jagged staircase. This non-convexity makes standard gradient descent impossible. As feature sets expand from dozens to thousands, traditional search algorithms like coordinate ascent frequently get trapped in poor local optima. Furthermore, standard $L_1$ or $L_2$ regularizers fail because MERT is scale-invariant: the model can bypass penalties by simply scaling all weights toward zero without changing the resulting translations.

The authors propose a "Regularized MERT" framework that addresses both the objective function and the optimization strategy.

Scale-Insensitive Regularization

To prevent the "scaling to zero" loophole, three approaches were explored:

  • $L_0$ Regularization: This penalizes the number of non-zero features rather than their magnitude, making it naturally immune to scaling tricks.
  • Modified $L_2$: The authors propose penalizing the distance from a known "good" baseline weight vector ($||w - w_0||_2$) or fixing one weight to a constant to remove a degree of freedom.
  • Efficient Line Search: The classic Och (2003) line search algorithm was adapted to handle these new penalties. While standard MERT search deals with flat steps, the regularized versions must handle quadratic curves ($L_2$) or discrete jumps ($L_0$).
Comparison of standard MERT, MERT with $L_2$ regularization, and MERT with $L_0$ regularization along a single search direction.

Gradient-Based Direction Finding

Instead of relying on random directions or coordinate ascent, the authors utilize the gradient of "expected BLEU." Expected BLEU provides a smooth, differentiable approximation of the jagged BLEU score. This gradient is used to determine the search direction, after which MERT’s exact, globally optimal line search is performed along that path. This hybrid approach combines the stability of smooth optimization with the precision of direct metric optimization.

Experimental Results

The approach was evaluated on synthetic data and real-world translation tasks with up to 3,600 features.

Scaling to High Dimensions

On a synthetic task scaling from 20 to 1,000 features, standard MERT failed to find the global optimum. In contrast, the new gradient-based MERT recovered the gold weight vector perfectly. As shown below, the expected BLEU gradient method maintains high performance (both in terms of BLEU and cosine similarity to the gold weights) even as the feature count grows, whereas traditional methods like Powell's or coordinate ascent degrade significantly.

Performance of different search algorithms as the number of features increases, showing BLEU scores and cosine similarity to the gold weight vector.

Real-World Performance

On translation tasks with 3,600 features (e.g., SparseHRM features for Chinese-English), Regularized MERT achieved BLEU scores comparable to Pairwise Ranking Optimization (PRO). Adding $L_2$ regularization provided a consistent boost of approximately 0.5 BLEU points over unregularized MERT, confirming that overfitting was a significant issue in the original formulation.

Conclusion

This work demonstrates that MERT's limitations in high-dimensional spaces are primarily due to its search strategy rather than its objective function. By introducing scale-insensitive regularization and a gradient-based direction finding algorithm, MERT can be successfully scaled to thousands of features. While the best-performing $L_2$ variant requires a good initialization and calculating the gradient adds some computational overhead, these enhancements revitalize a classic technique for modern MT systems.

Summary updated

Abstract

Minimum Error Rate Training (MERT) remains one of the preferred methods for tuning linear parameters in machine translation systems, yet it faces significant issues. First, MERT is an unregularized learner and is therefore prone to overfitting. Second, it is commonly used on a noisy, non-convex loss function that becomes more difficult to optimize as the number of parameters increases. To address these issues, we study the addition of a regularization term to the MERT objective function. Since standard regularizers such as ‘2 are inapplicable to MERT due to the scale invariance of its objective function, we turn to two regularizers—‘0 and a modification of‘2— and present methods for efficiently integrating them during search. To improve search in large parameter spaces, we also present a new direction finding algorithm that uses the gradient of expected BLEU to orient MERT’s exact line searches. Experiments with up to 3600 features show that these extensions of MERT yield results comparable to PRO, a learner often used with large feature sets.
BibTeX
@inproceedings{galley2013regularized,
  author = {Michel Galley and Chris Quirk and Colin Cherry and Kristina Toutanova},
  title = {{Regularized Minimum Error Rate Training}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  url = {https://openreview.net/forum?id=Bk4L9Mf_-B},
}

Loading related content...

Figure