Abstract
This paper introduces a new method for identifying named-entity (NE) transliterations in bilingual corpora. Recent works have shown the advantage of discriminative approaches to transliteration: given two strings (ws, wt) in the source and target language, a classifier is trained to determine if wt is the transliteration of ws. This paper shows that the transliteration problem can be formulated as a constrained optimization problem and thus take into account contextual dependencies and constraints among character bi-grams in the two strings. We further explore several methods for learning the objective function of the optimization problem and show the advantage of learning it discriminately. Our experiments show that the new framework results in over 50% improvement in translating English NEs to Hebrew.
Introduction
Transliteration—the process of phonetically mapping names between languages—is a critical hurdle for machine translation and information retrieval, especially when dealing with "out-of-vocabulary" words like names of people or places. The challenge lies in the fact that phonetic mapping is rarely a simple one-to-one lookup. For instance, in English-to-Hebrew transliteration, vowels are frequently omitted in the target script, and a single character in one language might map to multiple characters in another depending on the context.
Previous discriminative approaches treated this as a binary classification task: given two strings, does string B represent string A? However, these models often struggled because they evaluated character matches in isolation, ignoring the global structural legitimacy of the overall alignment.
Methodology
The authors propose a framework that shifts from local decisions toward global inference. Their approach consists of three main stages:
- Initial Representation (All-Features): The process begins with a noisy baseline that pairs all possible character n-grams between two strings if they appear in similar relative positions. This creates a dense graph of potential mappings.
- Inferring Informative Representations (IF): Instead of using all noisy features, the model employs Integer Linear Programming (ILP) to find the most "legitimate" alignment. By imposing constraints—such as requiring each character to map to at most one other character or an empty string—the feature set is pruned into a clean, structured graph.
- Discriminative Training: Rather than simply counting character match frequencies (a generative approach), the authors use a regularized average perceptron (via the SNoW learning architecture) to learn weights that distinguish correct transliterations from false ones.
The final decision model (Model 5) uses these discriminatively learned weights to drive the ILP optimization, ensuring the selected alignment is both structurally sound and phonetically probable.
Experimental Setup
The study focused on English-Hebrew transliteration, a difficult pair due to the lack of vowels in Hebrew script and differing character sets.
- Training Set: 250 English-Hebrew transliteration pairs.
- Test Set: 300 English-Hebrew transliteration pairs.
- Evaluation Metric: Mean Reciprocal Rank (MRR), which measures the rank of the correct transliteration among candidates.
Key Insights and Results
The core argument of the paper is that representation is paramount. By using ILP to "clean up" the feature set before training, the model focuses on meaningful phonetic alignments rather than random noise. While traditional models make "local" decisions (e.g., does 'a' map to 'א'?), this framework makes a "global" decision on whether the entire set of mappings is coherent.
A significant "aha" moment was the implementation of length normalization, dividing the final score by the length of the shorter word. Since Hebrew names are typically shorter, this prevents bias against longer, more complex names.
The results demonstrated a massive leap over existing baselines:
- Baseline (Previous State-of-the-Art): 0.51 MRR.
- Proposed Model (Model 5): 0.848 MRR.
- Model 5 + Length Normalization: 0.894 MRR.
The framework correctly identified the transliteration nearly 90% of the time and proved highly sample-efficient, reaching high accuracy with as few as 50–100 training samples.
Conclusion
While the principles of ILP and constrained optimization are universal, the authors acknowledge that different language pairs (like English-Chinese) might require distinct structural constraints. Future work could explore "soft constraints"—penalizing unlikely alignments rather than strictly forbidding them—to allow for greater flexibility in rare linguistic cases. Despite these areas for growth, the 50% performance boost marks a significant advancement in the field of named-entity transliteration.
Summary updated