A Structured Prediction Approach for Statistical Machine Translation

Dakun Zhang, Le Sun, Wenbo LiIJCNLP 2008 · OpenReview

Introduction

Traditional Statistical Machine Translation (SMT) systems, particularly phrase-based models, rely heavily on massive translation tables containing memorized word sequences. While effective, these models often lack a deep understanding of linguistic structure, specifically regarding reordering and nesting. Conversely, syntax-based models that utilize tree-to-tree transformations offer theoretical advantages but are frequently hindered by extreme computational complexity and the need for sophisticated linguistic parsers.

This paper introduces Amasis, a novel approach that bridges the gap between these two paradigms. By treating translation as a structured prediction problem—specifically a sequence labeling task over stack-based parsing operations—the researchers achieve translation quality comparable to state-of-the-art systems while drastically reducing the required model size.

Methodology: The Amasis Model

The core innovation of Amasis lies in reimagining translation as a series of stack operations within an Inversion Transduction Grammar (ITG) framework.

Figure 1: The Amasis system architecture, illustrating the integration of ITG-like trees and stack-based operations within a structured prediction framework.

ITG-like Trees and Stack Operations

The model uses ITGs to represent bilingual sentence pairs. These trees allow for both monotonic ("in-order") and non-monotonic ("reversed") alignments, effectively capturing cross-lingual word order variations. The construction of these trees is driven by three primary stack operations:

  • Shift (S): Pushing a word pair onto the stack.
  • Reduce in order (R+): Combining the top two stack items in their original sequence.
  • Reduce in reverse (R-): Combining the top two stack items in reverse order.

Structured Prediction with SEARN

By grouping these operations into labels (e.g., "S, R+"), the complex task of building a translation tree is reduced to a sequence labeling problem. The model employs the SEARN algorithm, a structured prediction framework that integrates search and learning. SEARN treats the process as a cost-sensitive classification task, allowing the model to learn from its own search errors during the training phase. During inference, a left-to-right beam search decoder identifies the optimal translation by maximizing the joint probability of the target sentence and the predicted structure.

Experimental Setup

The Amasis model was evaluated using the WMT06 corpus, focusing on three European language pairs: French-English (F-E), German-English (G-E), and Spanish-English (S-E). The experiments included both in-domain and out-of-domain test sets to assess the model's robustness and generalization capabilities.

Results and Discussion

The most significant finding is the extreme efficiency of the structured approach. By predicting the underlying structure of the translation rather than relying on exhaustive phrase lookups, Amasis achieves a massive reduction in translation table size.

Figure 4: Comparison of translation table sizes between the baseline Pharaoh system and Amasis. Amasis uses approximately 10% of the entries required by Pharaoh across all language pairs.

As shown in Figure 4, the Amasis translation table for French-English was only 9.68% the size of the baseline Pharaoh system's table. For Spanish-English, this reduction was even more pronounced, with the table size dropping to just 8.12%.

Despite these leaner tables, Amasis maintained competitive translation quality. In the French-English in-domain test, Amasis achieved a BLEU score of 27.44, significantly outperforming Pharaoh when limited to short phrases (20.54) and approaching the performance of Pharaoh with full phrase tables (30.01). Furthermore, the model demonstrated superior flexibility at the word level; over 30% of its table consisted of single-word pairs, compared to less than 5% for Pharaoh, highlighting its ability to dynamically assemble coherent structures from individual components.

Conclusion and Future Work

Amasis demonstrates that incorporating linguistic structure through structured prediction can lead to highly efficient SMT systems. By bypassing the need for massive phrase tables and complex linguistic parsers, the model offers a computationally lean alternative to traditional syntax-based methods. Future research will explore the integration of richer features, such as Part-of-Speech (POS) tags, and the use of more sophisticated classifiers within the SEARN framework to further close the performance gap with the best phrase-based baselines.

Summary updated

BibTeX
@inproceedings{zhang2008structured,
  author = {Dakun Zhang and Le Sun and Wenbo Li},
  title = {{A Structured Prediction Approach for Statistical Machine Translation}},
  booktitle = {IJCNLP},
  year = {2008},
  url = {https://openreview.net/forum?id=PoMuMxAP4U},
}

Loading related content...

Figure