A Hybrid Morpheme-Word Representation for Machine Translation of Morphologically Rich Languages

Minh-Thang Luong, Preslav Nakov, Min-Yen Kan

Abstract

We propose a language-independent approach for improving statistical machine translation for morphologically rich languages using a hybrid morpheme-word representation. In this model, the basic unit of translation is the morpheme, but word boundaries are strictly respected throughout the translation process. Our framework extends the classic phrase-based model through word boundary-aware morpheme-level phrase extraction, minimum error-rate training (MERT) optimized for word-level BLEU, and joint scoring using both morpheme- and word-level language models. Evaluation on English-to-Finnish translation using the Europarl corpus demonstrates statistically significant improvements in both BLEU scores and human judgments over traditional word-based baselines.

Introduction

In Statistical Machine Translation (SMT), the "word" has traditionally served as the atomic unit of translation. While effective for languages with limited morphology like English, this approach falters when applied to morphologically rich languages such as Finnish. In these languages, a single word can encapsulate the information of an entire English phrase through complex suffixing and prefixing.

Treating these complex words as indivisible tokens leads to two primary challenges:

  1. Data Sparsity: The proliferation of unique word forms reduces the frequency of individual tokens, making it difficult for the system to learn reliable translation parameters.
  2. The Unknown Word Problem: Systems often fail to translate inflected forms of a word even if the base form has been seen frequently during training.

Methodology

The proposed hybrid approach integrates morphological knowledge directly into the SMT decoding process without relying on language-specific hand-coded dictionaries.

Unsupervised Segmentation and Boundary Awareness

The system utilizes Morfessor for unsupervised morphological segmentation, splitting words into stems, prefixes, and suffixes. To maintain structural integrity, a special "+" notation tracks word boundaries (e.g., un/PRE+ care/STM+ ful/SUF+ ly/SUF). Crucially, the authors implemented Boundary-Aware Phrase Extraction. Unlike standard SMT, which extracts phrases of a fixed token length, this modified algorithm ensures that extracted morpheme sequences always correspond to complete word units, preventing the generation of illegal "gibberish" words.

The hybrid translation pipeline, integrating word-level and morpheme-level phrase tables (PT) for decoding.

Optimization and Scoring

To avoid the pitfalls of "morpheme-BLEU"—which can be high even if the resulting words are nonsensical—the system is optimized for final, reconstructed word-level BLEU during training. The decoder employs "Twin Language Models," simultaneously utilizing a morpheme-level model for local grammatical patterns and a word-level model for long-range context. Furthermore, phrase tables from word-based and morpheme-based systems are merged using a novel interpolation method to leverage the strengths of both representations.

Experimental Setup

The approach was evaluated on English-to-Finnish translation using the Europarl corpus, consisting of approximately 714,000 sentence pairs (15.5 million English words). Performance was tested across various data scales, from small subsets (40K pairs) to the full dataset, to assess the model's robustness under data scarcity.

Results and Discussion

The hybrid system demonstrated superior performance across all metrics. On the full dataset, it achieved a 0.74 BLEU point improvement over the standard word-based baseline, representing a statistically significant 5.6% relative gain.

Comparison of word-based (w-system) and morpheme-based (m-system) models across varying training set sizes.

As shown in the results, the morphological enhancements are particularly impactful on smaller datasets (T1-T4), where the m-system consistently outperforms the w-system in morpheme-level BLEU (m-BLEU), indicating better capture of morphological regularities. Human evaluations by native Finnish speakers further confirmed these findings, with a strong preference for the hybrid system's output, particularly in its ability to correctly inflect nouns and verbs.

Conclusion

By respecting word boundaries while operating at the morpheme level, this hybrid representation effectively mitigates data sparsity in morphologically rich languages. Future work will explore deeper integration of morphological analysis within the translation process and extension to other languages like Turkish or Czech.

Summary updated

Abstract

We propose a language-independent approach for improving statistical machine translation for morphologically rich languages using a hybrid morpheme-word representation where the basic unit of translation is the morpheme, but word boundaries are respected at all stages of the translation process. Our model extends the classic phrase-based model by means of (1) word boundary-aware morpheme-level phrase extraction, (2) minimum error-rate training for a morpheme-level translation model using word-level BLEU, and (3) joint scoring with morpheme- and word-level language models. Further improvements are achieved by combining our model with the classic one. The evaluation on English to Finnish using Europarl (714K sentence pairs; 15.5M English words) shows statistically significant improvements over the classic model based on BLEU and human judgments.
BibTeX
@inproceedings{luong2010hybrid,
  author = {Minh-Thang Luong and Preslav Nakov and Min-Yen Kan},
  title = {{A Hybrid Morpheme-Word Representation for Machine Translation of Morphologically Rich Languages}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2010},
  url = {https://openreview.net/forum?id=ex33-M9Kf8x},
}
Figure