Morphological Analysis for Statistical Machine Translation

Young-Suk LeeNAACL 2004

TL;DR

This paper introduces a novel morphological analysis technique to improve statistical machine translation (SMT) between languages with highly asymmetrical morphological structures, like Arabic and English. By inducing symmetry through fine-grained word segmentation and merging/deleting morphemes, the method significantly boosts translation quality for both word-based and phrase-based SMT models.

What Problem Does This Paper Address?

Statistical machine translation (SMT) systems often struggle when translating between languages with vastly different morphological complexities. Arabic, for instance, is morphologically rich, with prefixes and suffixes carrying significant grammatical information. This often leads to a single Arabic word corresponding to multiple English words, creating an imbalance in translation models. Standard SMT approaches, particularly word-based models, can fail to accurately capture these nuances, leading to degraded translation quality. The paper aims to bridge this gap by creating a more symmetrical representation of morphological information between Arabic and English for SMT.

Methods

The core of the proposed technique involves a two-step morphological processing for SMT:

  1. Word Segmentation: Arabic words are segmented into a sequence of prefix(es)-stem-suffix(es). This is a crucial first step to break down complex Arabic words into their constituent morphemes.
  2. Morphological Analysis (Merge/Deletion): This step identifies functional morphemes (prefixes and suffixes) that can be merged into their meaning-bearing stems or deleted. The goal is to induce a morphological symmetry with English, where such rich affixation is less common.
  3. For Word-Based Models (e.g., IBM Model 1): The algorithm uses part-of-speech (POS) tagging and Viterbi alignment on parallel corpora. It learns conditional translation probabilities between Arabic and English POS tags, and specifically between Arabic prefixes/suffixes and English POS tags. A key idea is to identify morphemes that don't have a clear, independent counterpart in English and are thus candidates for merging or deletion.
  4. For Phrase-Based Models: An additional, manually induced morphological analysis is applied, specifically focusing on the frequent Arabic determiner "Al#". This "Al#-deletion" aims to reduce the asymmetry caused by this common prefix.

Datasets

  • Training and Evaluation Corpora: The paper utilizes LDC-distributed Arabic Treebank and Penn Treebank for POS tagging. For training and evaluation, they use parallel corpora of varying sizes: 3.5K, 35K, 350K, and 3.3 million sentence pairs. These corpora are derived from AFP and Xinhua newswires.

Key Insights and Ideas

  • Morphological Symmetry is Key: The paper strongly advocates that achieving morphological symmetry between languages is crucial for improving SMT quality, especially when dealing with languages of differing morphological richness.
  • Automatic Induction of Merge/Deletion: The automatic induction of merge/deletion rules based on translation probabilities is a powerful way to identify and handle morphemes that don't have direct English equivalents.
  • "Al#" Determiner Asymmetry: The frequent use of the Arabic determiner "Al#" is identified as a significant source of asymmetry, and its manual deletion is shown to be beneficial for phrase-based models.
  • Impact Across Model Types: The proposed morphological analysis technique demonstrates significant improvements for both simpler word-based models (IBM Model 1) and more complex phrase-based translation models.

Key Results

The paper reports significant improvements in BLEU scores across various corpus sizes:

  • IBM Model 1: BLEU scores increased from a baseline of 0.10 to 0.25 (for 3.5K corpus) and from 0.18 to 0.32 (for 3.3M corpus).
  • Phrase Translation Model: BLEU scores improved from a baseline of 0.17 to 0.32 (for 3.5K corpus) and from 0.29 to 0.39 (for 3.3M corpus).
  • The paper notes that the manually induced "Al#-deletion" for phrase translation models, when combined with the automatically induced analysis, yielded better results than automatic induction alone.

Limitations and Future Work

  • Manual vs. Automatic Analysis: While the automatic induction of morphological analysis is effective, the "Al#-deletion" for phrase translation models was manually induced. Future work could explore more automated ways to handle such specific asymmetries.
  • Scope of Analysis: The paper focuses on Arabic-English. Extending this technique to other language pairs with similar morphological asymmetries could be a valuable direction.
  • Model Dependence: The effectiveness of the merge/deletion analysis is tied to the underlying SMT model. Further research could explore how these techniques interact with newer SMT architectures.

Summary updated

BibTeX
@inproceedings{lee2004morphological,
  author = {Young-Suk Lee},
  title = {{Morphological Analysis for Statistical Machine Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2004},
}
Figure