Abstract
We present an extension of phrase-based statistical machine translation models that enables the straight-forward integration of additional annotation at the word-level — may it be linguistic markup or automatically generated word classes. In a number of experiments we show that factored translation models lead to better translation performance, both in terms of automatic scores, as well as more grammatical coherence.
Introduction
Traditional phrase-based statistical machine translation (SMT) systems treat words as atomic, indivisible tokens. This approach fails to capture the relationship between different surface forms of the same lemma (e.g., "house" vs. "houses"), leading to significant data sparsity issues, particularly in morphologically rich languages like German, Czech, or Arabic. Furthermore, because these models lack explicit knowledge of syntax and morphology, they often produce grammatically incoherent output, such as mismatched gender or number in noun phrases.
Factored Translation Models
To address these limitations, we propose Factored Translation Models. In this framework, a word is represented as a vector of factors, which can include the surface form, lemma, part-of-speech (POS) tag, morphological features, or automatically generated word classes.
Mapping Steps
The translation process in a factored model is decomposed into a sequence of mapping steps. These are categorized into:
- Translation Steps: These map input factors to output factors at the phrase level (e.g., translating a source lemma to a target lemma).
- Generation Steps: These map factors within the output language at the word level (e.g., combining a target lemma and morphological features to generate the final surface word).
Multiple Language Models
A key advantage of the factored approach is the ability to utilize multiple language models (LMs) over different factors. While a high-order n-gram model on surface words is often computationally prohibitive due to sparsity, a 7-gram or 9-gram model on POS tags is highly effective. This acts as a "soft" grammar checker, penalizing ungrammatical sequences of word categories and enforcing better sentence structure.
Experimental Results
The framework was evaluated across several language pairs, demonstrating improvements in both translation quality and grammatical accuracy.
- Grammatical Coherence: In English-to-German translation, the factored model significantly improved noun-phrase agreement, reducing errors from 15% in the baseline to 4%.
- Translation Quality (BLEU):
- English-Czech: Incorporating morphology and POS tags led to a jump from 25.82 to 27.62 BLEU.
- English-Chinese: Adding automatic word classes as a factor increased BLEU from 19.54 to 21.10.
- Handling Sparsity: A "backoff" model for German-English, which uses lemma-based translation for rare words, outperformed the baseline by over 1.2 BLEU points.
Conclusion
Factored translation models provide a flexible framework for integrating linguistic and statistical information into SMT. By decomposing translation into mapping steps and utilizing factor-specific language models, we achieve better generalization and more grammatical output. Future work will focus on improving search efficiency and exploring asynchronous models where different factors can follow different phrase segmentations.
Summary updated