Abstract
We present a discriminative substring decoder for transliteration. This decoder extends recent approaches for discriminative character transduction by allowing for a list of known target-language words, an important resource for transliteration. Our approach improves upon Sherif and Kondrak's (2007b) state-of-the-art decoder, creating a 28.5% relative improvement in transliteration accuracy on a Japanese katakana-to-English task. We also conduct a controlled comparison of two feature paradigms for discriminative training: indicators and hybrid generative features. Surprisingly, the generative hybrid outperforms its purely discriminative counterpart, despite losing access to rich source-context features. Finally, we show that machine transliterations have a positive impact on machine translation quality, improving human judgments by 0.5 on a 4-point scale.
Introduction
In Machine Translation (MT), "Out-of-Vocabulary" (OOV) words present a significant challenge. When a system encounters an unknown word, it typically defaults to copying the source text or deleting it, leading to fragmented and unreadable outputs. In Japanese, this issue is particularly prevalent with Katakana, a script used primarily for foreign loanwords. Effectively "back-transliterating" these phonetic strings into their original English forms (e.g., furiimeeru to Freemail) is essential for high-quality translation. However, phonetic mapping is often ambiguous, and systems must ensure that the generated English words are valid and contextually appropriate.
Methodology
The researchers propose a discriminative substring decoder. Unlike traditional models that map individual characters, this decoder operates on character sequences or substrings, mirroring the phrasal approach used in Statistical Machine Translation (SMT).
The system utilizes several key technical components:
- Structured Perceptron: This learning engine discriminatively tunes model parameters to maximize 0-1 transliteration accuracy rather than just maximizing likelihood.
- Target Lexicon Integration: To prevent the generation of phonetically plausible but non-existent words, the decoder is grounded by a massive English lexicon (5 million word types). This acts as an anchor, steering the output toward valid English vocabulary.
- Feature Paradigm Comparison: The study compares two methods for data representation: Sparse Indicators (thousands of binary features capturing specific contexts) and Hybrid Generative Features (a small set of real-valued features from traditional models, weighted discriminatively).
Results and Discussion
The proposed system achieved a 28.5% relative improvement in accuracy over previous state-of-the-art methods. A critical finding was the efficacy of the "generative hybrid" approach. As shown in the ablation study, incorporating a target lexicon and generative features significantly boosted performance.
While the purely discriminative indicator approach allows for rich context, the distilled knowledge within traditional generative models proved highly potent when optimized within a discriminative framework. Furthermore, the integration of a target lexicon was a "game-changer," effectively filtering out gibberish. On the Wikipedia test set, the system's "Top 10" accuracy reached 65.6%, meaning the correct transliteration was present in the top ten candidates two-thirds of the time.
Beyond automated metrics, human evaluation revealed a substantial impact on translation quality. In a relative evaluation of 120 sentences, annotators preferred the transliteration-augmented MT output in 95 out of 120 cases, with human judgments improving by 0.5 points on a 4-point scale.
Conclusion
This work demonstrates that discriminative substring decoding, especially when combined with target-language lexicons and hybrid generative features, significantly advances the state-of-the-art in transliteration. While Katakana provides a clear signal for transliteration, future work will focus on developing script-agnostic classifiers to determine when to translate versus transliterate in more ambiguous language pairs like Arabic or Russian.
Summary updated