Abstract
When translating between languages with differing degrees of morphological synthesis, syntactic structures in one language may be realized as morphological structures in the other. Traditional Statistical Machine Translation (SMT) models often struggle with this mapping, relying on flat morpheme splitting that ignores hierarchical relationships. We propose a joint dependency model that integrates syntactic and morphological structures into a single tree representation. By modeling German compounds and particle verbs as hierarchical dependencies, our system achieves improvements of 1.4–1.8 BLEU in English–German translation tasks.
The Challenge: Compounds and Particle Verbs
Translating from a morphologically simple language like English to a complex one like German presents significant hurdles. Two primary issues are compounding and particle verbs:
- Compounding: German frequently combines multiple words into a single unit (e.g., Handgepäckgebühr for "carry-on bag fee"). Standard SMT models treat these as atomic units, limiting their ability to generate unseen compounds.
- Particle Verbs: Verbs like weggehen ("to go away") can appear as a single word or be split across a sentence depending on the grammatical context.
While previous work has attempted to split these words into morphemes, these approaches typically treat morphemes as a flat sequence. This loses the hierarchical structure necessary for learning morphosyntactic constraints, such as gender and case agreement.
The Joint Dependency Model
Our approach integrates morphological and syntactic structures into a unified dependency tree. This allows the translation engine to generalize at the level of morphemes while preserving the structural relationships that govern grammar.
Hierarchical Compound Representation
Instead of a flat list, compounds are represented as left-branching dependency trees. Crucially, the "head" of the compound—the part that determines the word's grammatical properties—is positioned to remain the parent of external modifiers, such as articles.
As shown in the figure above, the head gebühr (fee) is linked to the article eine, ensuring that the feminine gender of the head is respected, even if the preceding morphemes in the compound have different genders.
Particle Verb Abstraction
For particle verbs, we define a common representation that reorders the particle to always be the closest modifier of the verb. This abstraction allows the model to learn a single rule for the verb's meaning regardless of whether the particle is attached or separated in the surface text.
Technical Implementation: Head-Binarization
To make these dependency trees compatible with standard string-to-tree translation algorithms (like GHKM), we employ a novel head-binarization technique. Standard binarization often creates "virtual nodes" that break the linguistic hierarchy. Our method ensures that every node in the binarized tree still dominates the "head" of the constituent, allowing a Relational Dependency Language Model (RDLM) to accurately score the resulting structures during decoding.
Results and Impact
We evaluated our model on the WMT 2015 English–German translation task using 4.2 million sentence pairs for training.
- Translation Quality: The joint model outperformed the baseline by 1.4 to 1.8 BLEU points on the
newstest2014andnewstest2015benchmarks. - Grammatical Accuracy: In targeted tests for gender agreement, our model selected the correct article for rare compounds 96.5% of the time, compared to only 60-70% for a standard 5-gram language model.
- Productivity: The system successfully generated novel, grammatically correct compounds not seen in the training data, such as Staubsaugerroboter ("vacuum cleaner robot").
Conclusion
By treating the internal structure of words with the same hierarchical rigor as the structure of sentences, we bridge the gap between morphology and syntax in SMT. This joint modeling approach provides a robust framework for translating into morphologically rich languages, ensuring both lexical productivity and grammatical precision.
Summary updated