Abstract
Ming-Wei Chang, Dan Goldwasser, Dan Roth, Yuancheng Tu. Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
2009.
Introduction
Machine translation and cross-lingual search engines often struggle with Named Entities (NEs)—names of people, places, or organizations—because these words are frequently "out-of-vocabulary." Instead of being translated by meaning, they are transliterated based on sound. Traditional solutions rely on supervised machine learning, requiring thousands of manually labeled pairs. However, for many languages, such data is unavailable. Purely unsupervised methods often fail because phonetic mappings are context-dependent and noisy. This paper introduces Unsupervised Constraint Driven Learning (UCDL), a framework that bridges this gap by combining a noisy romanization table with logical linguistic constraints to bootstrap performance without labeled training data.
Methodology
The UCDL framework treats feature extraction as a structured prediction problem governed by constraints. The process involves three main stages:
- Initialization: The model begins with a standard, albeit noisy and incomplete, romanization table (e.g., from the Library of Congress) to establish initial character-level weights.
- Constraint-Driven Inference: The model identifies the optimal alignment between strings using two types of constraints:
- General Constraints: Universal rules such as Coverage (mapping every character) and No Crossing (preserving phonetic order).
- Language-Specific Constraints: Phonological rules (e.g., English 'p' mapping to 'b' or 'p' in Chinese) and word length restrictions.
- Iterative Self-Training: The model ranks potential transliteration candidates based on current weights. The top-ranked pair is labeled as a "positive" example, while others are "negative." A discriminative classifier (SVM) is then trained on these labels to refine weights, repeating until convergence.
Experimental Results
The researchers evaluated UCDL across three language families: Russian, Chinese, and Hebrew. The model significantly outperformed previous unsupervised and weakly supervised benchmarks.
Key Insights and Ablation
A key finding is that constraints can effectively substitute for labels. By giving the model the "rules of the game," it prunes the search space so effectively that it can learn phonetic nuances on its own. An ablation study highlighted this necessity: without constraints, the model's performance is negligible. However, as shown in the learning curves, the interaction between constraints and the iterative learning loop allows the model to rapidly improve its ranking capability, eventually rivaling supervised methods trained on hundreds of examples.
Conclusion
UCDL demonstrates that linguistic constraints can effectively substitute for labeled data in transliteration discovery. By using a discriminative approach and structural rules, the model avoids the out-of-vocabulary pitfalls of generative models. Future work may involve integrating temporal alignment data—such as the simultaneous appearance of names in news across languages—and exploring more complex, non-linear phonetic transformations.
Summary updated