Improvements in Phrase-Based Statistical Machine Translation

Richard Zens, Hermann NeyNAACL 2004

TL;DR

This paper introduces a phrase-based statistical machine translation (SMT) system that significantly improves translation quality and efficiency. By leveraging phrase-level translation models and a novel monotone search algorithm, the system achieves competitive or superior results compared to existing methods, particularly for language pairs with similar word order.

What Problem Does This Paper Address?

Traditional single-word based SMT systems struggle with capturing contextual information, leading to suboptimal translations. While phrase-based approaches offer a solution by translating sequences of words, they face challenges in efficiently searching for the best translation among a vast number of possibilities. This paper addresses these limitations by proposing a refined phrase-based SMT system with an efficient search algorithm and improved translation and smoothing models.

Methods

The core of the paper is the development of a phrase-based SMT system. Key methodological contributions include:

  • Phrase Extraction: A method for extracting bilingual phrases from aligned corpora based on a criterion that ensures words within a phrase are aligned only to each other.
  • Translation Model: A phrase-level translation model that decomposes the translation probability into source-to-target phrase probabilities.
  • Word-based Lexicon Smoothing: A technique to smooth phrase translation probabilities by incorporating a word-based lexicon, addressing the issue of sparse data for longer phrases. This uses a "noisy-OR" model and absolute discounting with interpolation.
  • Monotone Search Algorithm: A novel dynamic programming-based search algorithm that operates at the phrase level. This algorithm is "monotone," meaning it processes phrases in the order they appear in the source sentence, significantly reducing the search space and leading to linear complexity with respect to sentence length. This is a key innovation for efficiency.
  • Heuristics: Introduction of word and phrase penalties as feature functions to control sentence and phrase lengths, respectively.

Datasets

The paper evaluates the proposed system on three standard SMT tasks:

  • Verbmobil: German-English corpus for appointment scheduling, travel planning, and hotel reservation.
  • Xerox: Spanish-English corpus consisting of technical manuals.
  • Canadian Hansards: French-English corpus of Canadian parliamentary proceedings.

The authors provide statistics for training, development, and test sets for each corpus.

Key Insights and Ideas

  • Phrase-level translation is crucial: Moving beyond single words to phrases significantly improves translation quality by capturing local context.
  • Monotone search is surprisingly effective: Despite the potential for reordering within phrases, a monotone search at the phrase level is efficient and yields competitive results, especially for language pairs with similar word order. This suggests that large-scale reordering is not always necessary for good translation quality.
  • Lexicon smoothing is vital for phrase models: Pure relative frequencies for phrase translations are unreliable due to data sparsity. Smoothing with a word-based lexicon is essential for robust performance.
  • Efficiency gains are substantial: The monotone search algorithm leads to a dramatic increase in translation speed, making phrase-based SMT practical for large-scale applications.

Key Results

  • Improved Translation Quality: The phrase-based system consistently outperforms single-word based systems across all evaluated tasks and metrics (WER, PER, BLEU, NIST).
  • Competitive with Alignment Templates: The proposed system achieves results comparable to or better than the alignment template system, a strong baseline. For the Canadian Hansards task, the monotone phrase-based system even yields better results than the alignment template system.
  • Exceptional Efficiency: The monotone search algorithm achieves impressive translation speeds, with the Xerox task (over 10K words) taking less than 7 seconds and the Verbmobil task being even faster. For the Canadian Hansards task, translation of sentences up to 30 words takes about 1.5 seconds.

Limitations and Future Work

  • Monotonicity Constraint: The paper acknowledges that the monotone search might not be ideal for language pairs with significantly different word orders (e.g., German-English), where long-range reorderings are common. This is reflected in the lower "degree of monotonicity" for the Verbmobil task.
  • Future Work: The authors suggest investigating the usefulness of additional models, such as modeling segmentation probability. They also propose exploring ways to slightly relax the monotonicity constraint to allow for more reordering while still maintaining efficient search, potentially by allowing phrases to be skipped and translated later.

Summary updated

BibTeX
@inproceedings{zens2004improvements,
  author = {Richard Zens and Hermann Ney},
  title = {{Improvements in Phrase-Based Statistical Machine Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2004},
}
Figure