Abstract Standard phrase-based translation models do not explicitly model context dependence between translation units. As a result, they rely on large phrase pairs and target language models to recover contextual effects in translation. In this work, we explore n-gram models over Minimal Translation Units (MTUs) to explicitly capture contextual dependencies across phrase boundaries in the channel model. As there is no single best direction in which contextual information should flow, we explore multiple decomposition structures as well as dynamic bidirectional decomposition. The resulting models are evaluated in an intrinsic task of lexical selection for MT as well as a full MT system, through n-best reranking. These experiments demonstrate that additional contextual modeling does indeed benefit a phrase-based system and that the direction of conditioning is important. Integrating multiple conditioning orders provides consistent benefit, and the most important directions differ by language pair.
1. Introduction
In the landscape of 2013, Phrase-Based Statistical Machine Translation (PBSMT) was the dominant paradigm. However, PBSMT inherently treats translation units—phrases—as independent entities. This "independence assumption" creates a context gap at phrase boundaries. To mitigate this, systems traditionally relied on either extremely long phrase pairs, which suffer from data sparsity, or target language models, which only consider the output side and ignore source-side context. This paper addresses this limitation by explicitly modeling dependencies between adjacent units using n-gram Markov models over Minimal Translation Units (MTUs).
2. Methodology
The proposed framework moves beyond standard phrases to utilize Minimal Translation Units (MTUs). An MTU is defined as the smallest possible synchronized word grouping extracted from an alignment that provides a unique, non-overlapping segmentation of a sentence pair.
The core innovation of this work is the exploration of various Decomposition Structures to capture context:
- Traversal Orders: The authors explore four linear orders: Left-to-Right (L2R) and Right-to-Left (R2L), applied to both source and target sequences.
- Cyclic Models: These models condition each MTU on both its left and right neighbors simultaneously, similar to techniques used in POS tagging.
- Dynamic Decomposition: A sophisticated approach where the model dynamically selects the optimal traversal order for each sentence using a linear model with configuration features.
- Combination Strategies: "Product" models (multiplying probabilities from different orders) and "System Combination" (merging outputs) are used to leverage multiple directions.
3. Experimental Setup
The approach was evaluated across three diverse language pairs:
- Chinese-English (Chs-En): NIST datasets (approx. 1M sentence pairs).
- German-English (Deu-En): WMT 2006 data (approx. 751K sentence pairs).
- English-Bulgarian (En-Bgr): JRC-Acquis and TAUS data (approx. 4M sentence pairs), chosen for Bulgarian's complex morphology.
Evaluation involved an intrinsic "lexical selection" task and a full MT system integration via n-best reranking.
4. Results and Discussion
The experiments revealed that directionality is highly language-dependent. For instance, a right-to-left source order was most effective for Chinese-English, while a right-to-left target order performed best for German-English.
The "AllProduct" model, which combines multiple decomposition orders, consistently outperformed single-order models. This suggests that different traversal directions capture complementary contextual information. Crucially, performance gains were tied to looking across phrase boundaries; when restricted to intra-phrase context, the benefits disappeared.
In the full MT reranking task, significant improvements were observed:
- Chinese-English: +1.29 BLEU gain.
- English-Bulgarian: +2.82 BLEU gain, likely due to better resolution of morphological ambiguities.
- German-English: +0.55 BLEU gain.
5. Conclusion
This work demonstrates that explicitly modeling contextual dependencies across phrase boundaries significantly enhances translation quality. While dynamic decomposition showed promise, simpler product-based combinations of multiple traversal orders proved highly effective. Future research will focus on integrating these structures directly into first-pass decoding to improve computational efficiency and exploring more granular units like sub-word or dependency-tree structures.
Summary updated