TL;DR
This paper tackles the challenge of converting Japanese loanwords (written in katakana or romaji) back into their original English forms. They propose and evaluate two main approaches: direct transliteration and an indirect method using English pronunciation as an intermediate step. Their findings highlight the effectiveness of a lexicon-filter and a pair n-gram model for achieving the best back-transliteration performance.
What Problem Does This Paper Address?
Japanese has seen a significant influx of English loanwords, particularly since the 1990s, now making up a substantial portion of its vocabulary. While these loanwords are often written using the katakana script, the conversion process from English to Japanese is "information-losing." This means that the original English pronunciation and even spelling can be significantly altered, making it difficult to reverse the process – to accurately back-transliterate the Japanese loanword back to its English origin. This poses challenges for both Japanese speakers trying to understand the original English word and for computational systems aiming to perform this conversion. The paper aims to develop and evaluate methods for this back-transliteration task, exploring different input representations (katakana vs. romaji) and model architectures.
Methods
The authors propose two primary approaches for back-transliteration:
- Direct Transliteration: This method directly converts Japanese written forms (katakana or romaji) into their corresponding English words.
- Indirect Transliteration: This approach uses English pronunciation as an intermediate step. First, Japanese words are converted to their predicted English pronunciations, and then these pronunciations are used to predict the final English words.
They experiment with two types of models: those with and without an English lexicon-filter. This filter is designed to improve accuracy by ensuring that predicted English words are actual words present in an English lexicon.
Within each model type (with/without lexicon-filter), they implement three architectures:
- Pair n-gram model: This model uses WFSTs (Weighted Finite-State Transducers) to map Japanese characters to English characters and then computes a higher-order n-gram model. The output is a weighted lattice of possible English words, from which the shortest path is selected.
- LSTM-based sequence-to-sequence model: This employs a bidirectional LSTM encoder and a unidirectional LSTM decoder.
- Transformer-based sequence-to-sequence model: This architecture utilizes the standard transformer model with multiple encoder and decoder layers.
The novel aspect here is the incorporation of the lexicon-filter mechanism, particularly its application to both n-gram and sequence-to-sequence models.
Datasets
The dataset for this study was compiled from three sources:
- JMdict: A Japanese-English electronic dictionary used to extract katakana-English word pairs.
- CMUdict: Provides pronunciations for English words, used for both the indirect approach and building the English lexicon-filter.
- Webster's Dictionary: Used in conjunction with CMUdict to build a comprehensive English lexicon-filter.
The authors filtered out non-loanwords (like onomatopoeias) and manually expanded abbreviated katakana words (e.g., "アメフト" to "アメリカンフットボール"). The final dataset consists of 26,208 items, randomly split into 80% training, 10% development, and 10% testing sets. A notable characteristic is that 47.2% of the data involves mapping katakana words to multi-word English expressions.
Key Insights and Ideas
- Direct is Better: The direct transliteration approach consistently outperforms the indirect approach, suggesting that preserving the original written form (even if altered) is more beneficial than relying on potentially noisy phonetic conversions.
- Lexicon Filter is Crucial: The inclusion of an English lexicon-filter significantly improves performance across all models, demonstrating the importance of grounding predictions in actual English vocabulary.
- N-gram's Resilience: Surprisingly, the pair n-gram model, especially with a lexicon-filter, achieved the best performance, outperforming even the transformer models. This is noteworthy because transformer models have generally shown superior performance in many G2P and transliteration tasks.
- Input Representation Matters (Slightly): While the difference between using katakana and romaji as input was found to be insignificant, the paper explores both to understand their impact.
Key Results
The paper reports Word Error Rate (WER) as the primary evaluation metric, where lower WER indicates better performance.
- Best Performing Model: The pair n-gram model with a lexicon-filter, directly transliterating from katakana to English, achieved the lowest WER of 23.01.
- Lexicon Filter Impact: Models with lexicon-filters showed a significant reduction in WER, averaging a 10-point improvement compared to their no-lexicon-filter counterparts, especially in direct transliteration experiments.
- Direct vs. Indirect: The indirect approach performed worse than the direct approach across all model types.
- Katakana vs. Romaji: The difference in WER between using katakana and romaji as input was insignificant.
- Transformer vs. N-gram: While transformer models with lexicon-filters performed well in direct transliteration, the pair n-gram model surpassed them by 2-3 points in WER.
Limitations and Future Work
- Abbreviated Loanwords: A significant limitation is the handling of abbreviated Japanese loanwords (e.g., "スマホ" for "スマートフォン"). The current method manually expands these, but automating this process is a crucial area for future work due to the prevalence of such abbreviations.
- Spelling Errors: The paper identifies spelling errors, some due to phonetic distinctions between Japanese and English (e.g., "lighter" predicted as "writer") and others from English homophones (e.g., "site" vs. "sight").
- Word Delimiter Errors: Errors also occur in the placement of word delimiters, where words are correct but spaced incorrectly (e.g., "fireman" vs. "fire man"). These account for about 10% of errors and could be reduced by 2-3 points in WER if corrected.
Future work suggested includes integrating spelling correction models and incorporating word frequency computation. The authors also envision applying their model to challenges like machine translation and entity matching.
Summary updated