Learning Bilingual Linguistic Reordering Model for Statistical Machine Translation

Han-Bin Chen, Jian-Cheng Wu, Jason S. ChangNAACL 2009

TL;DR

This paper introduces a novel reordering model for statistical machine translation (SMT) that leverages bilingual linguistic features and considers phrase lengths. By addressing data sparseness and improving feature representativeness, their model significantly outperforms existing phrase-based and BTG-based SMT systems on Chinese-to-English translation tasks.

What Problem Does This Paper Address?

Statistical Machine Translation (SMT) systems, particularly those based on Bracketing Transduction Grammars (BTG), often struggle with accurately reordering words and phrases in the target language. Existing reordering models, while improving over basic distortion models, face challenges due to data sparseness and a lack of representative features. For instance, relying solely on the first or last words of phrases as features can lead to ambiguity and poor generalization, especially when dealing with similar but not identical words. This paper aims to tackle these issues by developing a more linguistically informed and robust reordering model.

Methods

The core of this work is a Maximum Entropy (ME) reordering model that incorporates bilingual linguistic features. To combat data sparseness, the authors employ:

  • Word Classification: Source language words are tagged with Part-of-Speech (POS) using the CKIP system, and target language words are classified using the mkcls toolkit. This allows for generalization across similar words and avoids issues with phrase table sparsity.
  • Phrase Length Consideration: Features are extracted considering different combinations of phrase lengths (M, L, R, B classes). This helps to distinguish between cases that might appear similar based on word identity alone but have different reordering behaviors due to their length.
  • Bilingual Features: The model utilizes features derived from both source and target language information, aiming for a more comprehensive understanding of reordering needs.

The system is built upon the BTG framework and uses a bottom-up CKY-style decoder.

Datasets

  • Training Data: 2.2 million sentence pairs from the Hong Kong Parallel Text (LDC2004T08) and Xinhua News Agency (LDC2007T09) corpora, filtered for sentences shorter than 60 words.
  • Language Model: Xinhua news from English Gigaword Third Edition (LDC2007T07) for building a trigram model.
  • Development Set (for tuning): NIST MT-08 test set, with sentences no more than 20 words long.
  • Test Set: NIST MT-06 test set for Chinese-to-English translation.

Key Insights and Ideas

  • Linguistic Features are Crucial: Simply using lexical features (like first/last words) is insufficient and leads to data sparseness. Incorporating linguistic information like POS tags and word classes is key to generalization.
  • Phrase Length Matters: The length of phrases significantly impacts reordering. Treating phrases of different lengths as distinct cases leads to more accurate predictions.
  • Bilingual Context is Powerful: Leveraging information from both source and target languages simultaneously provides a richer feature set for reordering.
  • Combining WC and LC is Essential: While Word Classification (WC) and Length Consideration (LC) individually improve performance, their combination yields the most significant gains, suggesting they address complementary aspects of the reordering problem.

Key Results

The proposed bilingual linguistic reordering model achieved the following improvements:

  • Outperformed the baseline phrase-based Moses system by 2.41 BLEU points.
  • Outperformed the baseline BTG-based SMT system by 1.31 BLEU points.
  • The model achieved a BLEU score of 24.96, significantly higher than MEBTG (23.65) and Moses (22.55).
  • The proposed model uses a significantly smaller feature set (1.41K features) compared to MEBTG (171K features) while achieving better performance, indicating improved efficiency and generalization.

Limitations and Future Work

  • Future Linguistic Classification: The authors suggest investigating more sophisticated linguistic ways to classify words, particularly for the target language. They propose exploring hierarchical structures in WordNet for richer semantic information.
  • Generalization: While the current work improves generalization through classification, further research into methods for generalizing reordering models is encouraged.
  • Chinese Parsers: The paper notes that current Chinese parsers have lower accuracy compared to English ones, which is why they opted for a more reliable word segmentation system. Future work could potentially benefit from improved Chinese parsing technologies.

Summary updated

BibTeX
@inproceedings{chen2009learning,
  author = {Han-Bin Chen and Jian-Cheng Wu and Jason S. Chang},
  title = {{Learning Bilingual Linguistic Reordering Model for Statistical Machine Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
}
Figure