Statistical machine translation (SMT) systems generally excel at local lexical mapping—finding the right word or short phrase in a target language. However, they frequently struggle when the "blueprint" of the source language differs fundamentally from the target. This phenomenon, known as structural divergence, occurs when languages follow different word-order rules, such as the Subject-Verb-Object (SVO) pattern of English versus the Subject-Object-Verb (SOV) pattern of Japanese or Hindi.
When these blueprints clash, standard translation models often produce garbled or incoherent output because they cannot easily "jump" words across the long distances required by the target syntax.
The Alignment Bottleneck
The core of the problem lies in word alignment, the process of determining which source words correspond to which target words during training. Many efficient alignment models, such as fast_align, operate on the assumption that word order remains relatively consistent between languages. As Ding et al. note, this "distortion-blind" assumption fails when processing pairs like SVO and SOV, where verbs or prepositional phrases must move significantly.
These structural mismatches create "jumping" dependencies that are difficult for standard systems to map correctly. For example, in Modern Standard Arabic, the Verb-Subject (VS) construction is dominant, appearing in roughly 70% of cases. Carpuat et al. argue that this divergence from English's Subject-Verb (SV) order causes models to treat identical semantic relationships as distinct patterns, leading to data sparsity and reduced accuracy. Similarly, Cherry et al. show that structural divergence even complicates non-textual tasks like subtitle timing, as the accuracy of mapping segment boundaries is highly sensitive to word order differences.
Preordering: Making the Source Look Like the Target
To bridge this gap, researchers developed a strategy called preordering (or pre-reordering). The intuition is simple: before the translation engine begins its work, the source sentence is rearranged to match the syntactic structure of the target language.
Early successful interventions relied on hand-crafted linguistic rules. Langlais et al. demonstrated that rule-based reordering of German verbs significantly improved translation quality by resolving the mismatch between German's verb-final and English's verb-medial structures. For English-to-Japanese translation, Duh et al. introduced "Head Finalization," a simple rule that moves the head of a phrase to the end to mimic Japanese syntax.
However, manual rules can be insufficient for deep structural differences. Yeniterzi et al. found that simply reordering constituents was not enough for English-to-Turkish translation; they instead mapped English function words (like "on" or "my") into morphological suffixes to match Turkish's agglutinative structure.
Learning Structural Transformations
As the field evolved, researchers moved from manual rules to automated methods for learning reordering patterns from data. P. V. S. Avinesh proposed using Association Rule Mining to automatically discover syntax-based reordering rules from parallel corpora. Other approaches leveraged more complex machinery:
- Discriminative Classifiers: Lerner et al. used dependency parse trees and lexical features to train classifiers that handle long-distance reorderings, which is particularly effective for highly divergent pairs like English-Japanese.
- Neural Networks: de Gispert et al. utilized neural networks to handle the complexity of reordering operations, providing a faster and more accurate alternative to traditional search-based reordering during decoding.
- Dependency-Based Models: Huang et al. combined dependency relations with alignment data to improve Chinese-to-Vietnamese translation, specifically targeting the conflicting placements of prepositional and noun phrases.
These automated methods are not without risks. Han et al. highlight that parsing errors—such as misclassifying a verb or failing to identify the root of a sentence—can severely degrade the performance of syntax-based preordering.
Syntactic Consensus in System Combination
The challenge of word order mismatch extends beyond individual models to the task of system combination, where multiple translation outputs are merged. Because different decoders utilize distinct reordering logic, their outputs often vary wildly in structure. Huang et al. and Chen et al. note that when alignment algorithms fail to reconcile these divergent structures, the resulting "confusion network" produces garbled text.
To solve this, Watanabe et al. proposed using a Confusion Forest. Instead of aligning simple word strings, this method encodes translation hypotheses into a forest of parse trees. By seeking a "syntactic consensus" rather than just surface-level similarity, the system can synthesize a new translation that is more grammatically coherent than any of the individual inputs.
Key Foundational Work
RALI: SMT Shared Task System Description (2005) — contributed 1 concept elements to this direction.
Structural Feature Selection For English-Korean Statistical Machine Translation (2000) — contributed 1 concept elements to this direction.