Consensus Training for Consensus Decoding in Machine Translation

Adam Pauls, John DeNero, Dan KleinEMNLP 2009 · OpenReview

Abstract

We propose CoBLEU (Consensus BLEU), a novel training objective for discriminatively tuning log-linear machine translation models. Unlike standard Minimum Error Rate Training (MERT), which optimizes for the single best Viterbi path, CoBLEU optimizes the BLEU score of expected n-gram counts across the entire translation forest. This continuous objective allows for stable gradient-based optimization and provides natural regularization, leading to improved generalization and superior performance when paired with consensus decoding methods.

Introduction

In statistical machine translation (SMT), models are traditionally tuned using Minimum Error Rate Training (MERT). MERT seeks to find feature weights that maximize the BLEU score of the highest-scoring translation candidate. However, this approach suffers from two primary drawbacks. First, there is a mismatch between training and inference: modern decoders often employ "consensus decoding" (or Minimum Bayes Risk decoding), which considers n-gram frequencies across multiple likely translations rather than just the top path. Second, MERT's objective function is non-smooth and "jagged," making it prone to instability and severe overfitting on small tuning sets.

The CoBLEU Objective

To address these issues, we introduce CoBLEU. Instead of calculating BLEU based on the n-gram counts of a single Viterbi derivation, CoBLEU utilizes the expected counts of n-grams across a "packed forest" representing an exponential number of possible translations.

Comparison of the smooth CoBLEU objective (CoBP) against the discontinuous MaxBP objective used in standard tuning.

As illustrated in the figure above, the standard max-BLEU objective (MaxBP) is a step function that changes abruptly as different hypotheses become the top-scoring candidate. In contrast, the consensus objective (CoBP) is smooth and differentiable, enabling the use of standard gradient ascent for optimization. This smoothness acts as a natural regularizer; by rewarding the model for placing probability mass on many "good" n-grams, the resulting weights are less sensitive to the specific quirks of the tuning data.

Technical Innovation: Gradient Computation in Forests

Optimizing CoBLEU requires calculating the gradient of the expected BLEU score. This involves computing the "expectation of products"—specifically, the product of feature counts and n-gram counts—over the massive translation forest. We present a novel dynamic program, an extension of the inside-outside algorithm, that performs this computation in linear time relative to the forest size. This allows us to perform forest-based tuning efficiently, moving beyond the limitations of k-best lists.

Experimental Results

We evaluated CoBLEU on Spanish-English and French-English tasks using the Europarl corpus. Our results demonstrate that CoBLEU-trained models consistently outperform MERT when using consensus decoding at test time.

Optimization stability: CoBLEU exhibits smooth convergence compared to the erratic behavior of MERT.

A key advantage of CoBLEU is its stability. As shown in the convergence plot, MERT's objective value fluctuates significantly during training as new k-best lists are generated. CoBLEU, however, follows a smooth and predictable path to convergence. Furthermore, CoBLEU significantly reduces overfitting. In Spanish-English experiments, MERT-trained models experienced a 2.3 BLEU point drop from tuning to test sets, whereas CoBLEU-trained models showed much greater robustness. Notably, CoBLEU even improved performance for standard Viterbi decoding in some cases, highlighting its superior generalization capabilities.

Conclusion

CoBLEU provides a principled framework for training SMT models that aligns with modern decoding practices. By optimizing over the entire distribution of translations, it achieves greater stability, better generalization, and higher translation quality. While the objective currently favors slightly shorter sentences—suggesting room for refinement in the expected length component—the use of packed forests and efficient dynamic programming makes CoBLEU a viable and robust alternative to traditional tuning methods.

Summary updated

Abstract

We propose a novel objective function for discriminatively tuning log-linear machine translation models. Our objective explicitly optimizes the BLEU score of expected n-gram counts, the same quantities that arise in forest-based consensus and minimum Bayes risk decoding methods. Our continuous objective can be optimized using simple gradient ascent. However, computing critical quantities in the gradient necessitates a novel dynamic program, which we also present here. Assuming BLEU as an evaluation measure, our objective function has two principle advantages over standard max BLEU tuning. First, it specifically optimizes model weights for downstream consensus decoding procedures. An unexpected second benefit is that it reduces overfitting, which can improve test set BLEU scores when using standard Viterbi decoding.
BibTeX
@inproceedings{pauls2009consensus,
  author = {Adam Pauls and John DeNero and Dan Klein},
  title = {{Consensus Training for Consensus Decoding in Machine Translation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
  url = {https://openreview.net/forum?id=ByVIwGfObH},
}

Loading related content...

Figure