Phrase Training Based Adaptation for Statistical Machine Translation

Saab Mansour, Hermann NeyNAACL 2013

TL;DR

This paper introduces a novel phrase training method to adapt general-domain statistical machine translation (SMT) models to specific domains, like lecture transcripts. By leveraging forced alignment on in-domain data, they effectively bias phrase probabilities towards the target domain, achieving significant improvements over standard methods and even outperforming mixture modeling approaches.

What Problem Does This Paper Address?

Statistical machine translation (SMT) systems are often trained on large, general-domain datasets (e.g., news, parliamentary proceedings). However, their performance can degrade significantly when applied to specialized domains with different vocabulary and linguistic styles (e.g., lectures, medical texts). Domain adaptation aims to bridge this gap by leveraging smaller amounts of in-domain data to improve translation quality. Existing methods often involve weighting different parts of the translation model or interpolating models, but this paper tackles the problem by directly adapting the phrase translation probabilities themselves through a specialized phrase training procedure.

Methods

The core of this paper is a novel adaptation technique called Phrase Training Based Adaptation. Here's how it works:

  • Standard Phrase Training: Typically, phrase training involves using a decoder to re-estimate phrase probabilities from aligned data. This paper uses a forced alignment (FA) method for this, which is known to be robust.
  • Adaptation Twist: Instead of training on the same data used to extract the initial phrase table, the authors initialize with a general-domain phrase table and then perform the FA phrase training specifically on the in-domain data. This process is denoted as Y'-FA-IN, where Y' is the general domain data and IN is the in-domain data.
  • Key Idea: This approach directly biases the probabilities of the general phrase table towards the in-domain distribution, rather than just weighting existing phrases. They also explore variations, such as adapting an "other-domain" (OD) table using in-domain data (OD-FA0-IN).
  • Comparison: They compare their method against standard heuristic phrase extraction and standard FA training, as well as mixture modeling techniques.

Datasets

The experiments were conducted on the IWSLT 2011 TED translation tasks, specifically:

  • Arabic-to-English:
    • In-domain (IN): ~100K TED talk sentences.
  • Other-domain (OD): ~8M United Nations (UN) sentences.
  • German-to-English:
    • In-domain (IN): ~130K TED talk sentences.
  • Other-domain (OD): ~2.1M sentences from news-commentary and europarl corpora.

They also used additional monolingual data for language model training. The datasets are characterized by their size and domain specificity.

Key Insights and Ideas

  • Direct Probability Adaptation is Powerful: Directly re-estimating phrase probabilities using in-domain data via forced alignment is more effective than simply weighting existing phrases or interpolating models.
  • Leveraging General Data: Even when starting with a general-domain phrase table, adapting it with in-domain data can significantly improve performance, especially for domains where general data might not fully cover specialized terminology.
  • Phrase Table Size Reduction: A notable side effect of their FA adaptation method is a significant reduction in the size of the adapted phrase table, which can lead to faster decoding.
  • Context Matters for OD Data: The usefulness of "other-domain" data varies. For Arabic-English, it was less helpful than for German-English, suggesting that the domain discrepancy plays a crucial role.

Key Results

The proposed phrase training adaptation method (specifically ALL-FA-IN) consistently outperformed baselines:

  • Arabic-to-English: The adapted system achieved improvements over the unadapted general system, closing about half the gap between the general and in-domain systems. For example, on the eval set, they saw a +0.4% BLEU and -0.6% TER improvement compared to the best heuristic system.
  • German-to-English: The adapted system showed improvements, particularly in TER. For instance, on the test set, they observed a -0.8% TER improvement.
  • Comparison to Mixture Modeling: Their adapted phrase tables, when used in a mixture modeling framework, consistently yielded better results than using heuristically extracted OD tables.

Limitations and Future Work

  • Limitations: The paper notes that for Arabic-to-English, the OD data's usefulness was limited, and interpolation was more beneficial. This suggests that the degree of domain discrepancy and the size of the OD data can influence the effectiveness of adaptation strategies.
  • Future Work: The authors suggest further exploration of mixture modeling, particularly by incorporating their phrase training adapted OD tables. They also implicitly suggest that understanding the interplay between domain discrepancy, data size, and adaptation method is an ongoing area of research.

Summary updated

BibTeX
@inproceedings{mansour2013phrase,
  author = {Saab Mansour and Hermann Ney},
  title = {{Phrase Training Based Adaptation for Statistical Machine Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}
Figure