Abstract
We present a discriminative model that directly predicts which set of phrasal translation rules should be extracted from a sentence pair. Our model scores extraction sets: nested collections of all the overlapping phrase pairs consistent with an underlying word alignment. Extraction set models provide two principle advantages over word-factored alignment models. First, we can incorporate features on phrase pairs, in addition to word links. Second, we can optimize for an extraction-based loss function that relates directly to the end task of generating translations. Our model gives improvements in alignment quality relative to state-of-the-art unsupervised and supervised baselines, as well as providing up to a 1.4 improvement in BLEU score in Chinese-to-English translation experiments.
Introduction
Traditional Statistical Machine Translation (SMT) systems rely on a pipeline where word alignments are first generated by unsupervised tools like GIZA++, followed by a heuristic extraction of phrasal translation rules. This "align-then-extract" approach suffers from a mismatch of objectives: word aligners optimize for Alignment Error Rate (AER), which may not correlate with the quality of the resulting translation rules. Furthermore, word-level models cannot leverage phrase-level information during the alignment process.
Methodology: Modeling Extraction Sets
To address these limitations, we introduce a discriminative model that directly predicts the extraction set—the complete collection of phrase pairs licensed by an alignment. By modeling the extraction set directly, we can incorporate features on phrase pairs, such as relative frequency and lexical templates, alongside traditional word-link features.
Key Innovations
- Phrasal Features: The model incorporates features directly on phrase pairs, allowing it to evaluate the quality of the rules it will eventually produce.
- Inversion Transduction Grammar (ITG) Constraints: We restrict the search space to ITG alignments, enabling efficient dynamic programming for finding the optimal extraction set.
- Coarse-to-Fine Inference: To manage the computational cost of scoring overlapping phrases, a fast "coarse" pass prunes the search space before a more complex "fine" pass is applied.
- Recall-Biased Loss: The model is trained using MIRA, optimizing for a phrase-level F-measure ($F_5$) that favors recall to ensure useful translation rules are not missed.
Experimental Setup
The model was evaluated on Chinese-to-English translation tasks. Training and evaluation for alignment used a hand-aligned portion of the NIST MT02 set. Translation performance was tested on NIST MT04 and MT05 sets using a 22.1 million-word parallel corpus. We compared our approach against GIZA++ and a state-of-the-art Joint HMM aligner.
Results and Analysis
The discriminative extraction set model outperformed all baselines. It achieved an AER of 15.6, compared to 27.8 for GIZA++. On the Bispan $F_5$ metric, which evaluates phrase extraction quality, the model scored 74.0, a significant improvement over GIZA++'s 46.0.
The Role of Non-Lexical Links
A key insight of this work is the importance of "possible" links. Human annotators often mark links as "possible" when words are not direct lexical equivalents but serve similar roles. Our model leverages these to license overlapping phrases, providing flexibility in ambiguous regions.
These improvements translated directly to better end-task performance, yielding a 1.4 BLEU point gain with the Joshua hierarchical system and a 1.2 BLEU gain with Moses.
Conclusion
By shifting the modeling focus from individual word links to the set of extracted phrases, we align the training objective with the ultimate goal of machine translation. While the approach requires careful optimization and is currently constrained by ITG grammar, it demonstrates the clear benefit of discriminative, phrase-aware alignment models. Future work will explore extending this framework to more complex rule formalisms like syntactic grammars.
Summary updated