Introduction
Word alignment is a fundamental task in statistical machine translation (SMT), identifying correspondences between words in source and target sentences. While traditional generative models like the IBM series are widely used, they are inherently limited by a one-to-many architecture. This restriction struggles to capture the complex many-to-many relationships prevalent in natural language.
A common solution involves training bidirectional models (Source → Target and Target → Source) and enforcing an "agreement" constraint. However, standard agreement treats all words uniformly, which is suboptimal for linguistically distant pairs like Japanese and English. In these cases, function words often lack direct 1-to-1 mappings or are omitted entirely (pro-drop), making strict agreement counterproductive.
Frequency-Based Posterior Regularization
This research introduces a more nuanced approach using the Posterior Regularization (PR) framework. PR allows for the injection of prior linguistic knowledge into the Expectation-Maximization (EM) process without increasing the underlying model's complexity.
The core innovation is the Frequency Constraint, which uses word frequency as a proxy for word type. High-frequency words are categorized as function words (e.g., "the", "is"), while low-frequency words are treated as content words (e.g., "algorithm"). The authors propose three specific constraints:
- Mismatching Constraint ($f2c$): Penalizes alignments between a function word in one language and a content word in another.
- Matching Constraints ($f2f$ and $c2c$): Encourages agreement specifically when both directional models identify a function-to-function or content-to-content alignment.
Experimental Results
The method was evaluated on the French-English Hansard corpus, the Japanese-English Kyoto Free Translation Task (KFTT), and the NTCIR10 patent dataset.
Alignment and Translation Quality
The frequency-based constraints consistently outperformed the standard symmetric baseline. On the KFTT task, the $f2f$ constraint achieved an Alignment Error Rate (AER) of 47.73, a notable improvement over the 48.18 baseline. Furthermore, the NTCIR10 task showed statistically significant gains in BLEU scores, indicating that better word alignment directly translates to higher-quality machine translation.
Precision-Recall Trade-off
As shown in the figures above, the proposed constraints improve the precision-recall trade-off. By penalizing "garbage collection"—a phenomenon where rare words are incorrectly mapped to frequent function words just to satisfy model constraints—the model makes more accurate, linguistically sound alignment decisions.
Conclusion
By leveraging word frequency as a simple yet effective feature, this paper demonstrates that unsupervised word alignment can be significantly improved through targeted posterior constraints. This approach is particularly beneficial for language pairs with disparate grammatical structures. Future work may involve more sophisticated word-type identification, such as unsupervised part-of-speech induction, to further refine these constraints.
Summary updated