TL;DR
This paper introduces a novel method to enhance phrase-based statistical machine translation (SMT) by pre-generating high-quality, linguistically-rich phrase pairs. By leveraging "phrase prototypes" derived from POS tags and alignments, they enrich the decoder's options, leading to improved translation accuracy.
What Problem Does This Paper Address?
Statistical Machine Translation (SMT), particularly phrase-based approaches, often struggles to incorporate complex linguistic information directly into the decoding process due to computational constraints. While some methods use linguistic features before or after decoding, they are often limited in scope or computationally expensive. This paper addresses the challenge of enriching the phrase table used by SMT decoders with more linguistically informed phrase pairs, aiming to provide better translation options without overwhelming the decoder.
Methods
The core innovation is the use of "phrase prototypes." These prototypes are derived from bilingual phrase pairs by converting surface word forms into Part-of-Speech (POS) tags for aligned words, while keeping unaligned words as their surface form. This creates a more abstract representation that captures grammatical structure.
The process involves:
- Learning Phrase Prototypes: From a corpus of aligned, POS-tagged bilingual text, phrase pairs are extracted. Each aligned word/tag is converted into a POS tag, forming a prototype tuple (source prototype, target prototype, alignment prototype).
- Generating New Phrases: The system scans the source text to be translated. Any source span matching a learned phrase prototype triggers the generation of corresponding target phrases. This generation uses a word-to-word dictionary and honors the alignment prototype.
- Filtering and Scoring: Generated target phrases are POS-tagged. Only those matching the target prototype's POS sequence are retained. The phrase prototype itself is assigned a score based on its observed frequency in the training data (p(SP, AP|TP) * p(TP, AP|SP)). This score is then incorporated into the phrase table.
This approach differs from traditional methods by generating new phrase pairs in isolation using a richer model, rather than trying to integrate complex features directly into the sentence-level decoder.
Datasets
- Training Data: Linguistic Data Consortium (LDC) Arabic-English corpus (34K sentences, ~1 million words) was used for training the baseline MT system and learning phrase prototypes.
- Tuning and Testing Data: LDC Multi-translation Arabic-English corpus (NIST2003) was used. The tuning set consisted of the first 500 sentences, and the test set consisted of the next 500 sentences.
- Language Model Data: The English side of the parallel corpus, plus the English side of WMT07 German-English and French-English news commentary data.
- POS Tagging: Stanford Arabic POS Tagging system, based on Toutanova et al. (2003).
- Dictionary: Extracted from the highest-scoring translations for each source word in the baseline phrase table, with manual additions for some closed-class words.
Key Insights and Ideas
- Leveraging Linguistic Richness: The paper demonstrates that it's feasible and beneficial to use linguistically richer features (like POS tags) to generate phrase pairs before the main decoding step, overcoming the computational limitations of using them during decoding.
- Phrase Prototypes as Bridges: Phrase prototypes act as an effective abstraction, allowing the system to generalize from observed phrase pairs to generate new, potentially better ones.
- Language Model as a Filter: The language model plays a crucial role in filtering out poorly formed generated phrases, allowing the system to benefit from the generated options even if some are imperfect.
- Pre-computation for Decoder: The core idea of pre-translating short spans with a richer model to augment the decoder's phrase table is a practical and effective strategy.
Key Results
- BLEU Score Improvement: The proposed method achieved a 1.3 BLEU score increase (3.3% relative) on the Arabic-English test set compared to the baseline Moses system.
- Tuning Set Performance: On the tuning set, BLEU scores generally improved as the amount of bilingual training data for prototype extraction increased, with generated phrases approaching 100% of the phrase table.
- Baseline System Score: The tuned baseline Moses system scored 38.45 on the test set.
- Improved System Score: The system with generated phrases scored 39.75 on the test set.
Limitations and Future Work
- Unaligned Word Handling: The current method insists that unaligned words are always kept as their surface form in the target prototype. This simplifies generation but might miss opportunities for more abstract translations.
- Alignment Prototype Simplicity: The alignment prototype is restricted to 1:1 alignments, excluding more complex alignment types (1:many, many:1, many:many).
- Potential for Noise: As more training data is used, more phrase prototypes are learned, potentially introducing more noise into the phrase table. The language model helps mitigate this, but it's an ongoing trade-off.
- Future Work: The paper suggests that incorporating more sophisticated alignment models and exploring different ways to handle unaligned words could further improve the system. The authors also note that a 5-gram language model (vs. the 4-gram used) might better prefer correct phrases over shorter ones, indicating potential for LM improvements.
Summary updated