Zero-shot North Korean to English Neural Machine Translation by Character Tokenization and Phoneme Decomposition

Hwichan Kim, Tosho Hirasawa, Mamoru KomachiACL 2020

TL;DR

This paper tackles the challenge of translating North Korean to English, a task severely hampered by the scarcity of parallel data. The authors propose a novel zero-shot approach by leveraging similar South Korean data, tokenizing it at the character level and decomposing characters into phonemes. This method significantly improves translation accuracy, demonstrating the effectiveness of character-level processing and phoneme decomposition in bridging the grammatical gap between the two Korean dialects.

What Problem Does This Paper Address?

Machine translation (MT) systems, particularly Neural Machine Translation (NMT), thrive on large amounts of parallel text data for training. However, translating between North Korean and English presents a significant challenge due to the extreme scarcity of such data. This "low-resource" bottleneck makes achieving high translation accuracy for North Korean very difficult. Existing approaches to low-resource MT, such as using pivot languages or many-to-many translation models, have limitations when applied to this specific language pair. Furthermore, while North and South Korean are linguistically similar, they exhibit grammatical differences in areas like word segmentation and initial sound rules, which can further complicate direct translation efforts even when using South Korean data as a proxy.

Methods

The core of this paper's contribution lies in its innovative preprocessing of South Korean text before feeding it into an NMT model for translation to English. The authors propose two main tokenization strategies:

  • Character Tokenization: Instead of treating words as the basic units, sentences are split into individual characters. This approach helps mitigate differences in word segmentation (WS) between North and South Korean, as character-level units are more consistent.
  • Phoneme Decomposition (with Byte-Pair Encoding - BPE): Characters are further decomposed into their phonetic components (vowels and consonants). This method is particularly effective in addressing differences in the initial sound rule (ISR), where certain consonants change their form depending on the following vowel. By breaking down words into phonemes, the model can better handle these variations.

The paper explores combinations of these methods:

  • Word (character BPE) model: Standard word tokenization with character-level BPE applied to words.
  • Character model: Pure character-level tokenization.
  • Word (phoneme BPE) model: Words are decomposed into phonemes and then BPE is applied.
  • Character (phoneme BPE) model: Sentences are first tokenized at the character level, then characters are decomposed into phonemes, and finally, BPE is applied to the phoneme sequence. This is the proposed method that combines the benefits of both character and phoneme processing.

These preprocessed South Korean sentences are then used to train a North Korean to English translation model in a zero-shot manner.

Datasets

  • Training Data: A News Korean-English parallel corpus was used for training. This corpus was then converted to North Korean grammar for evaluation purposes.
  • Evaluation Data: A North Korean-English evaluation dataset was created by manually translating a South Korean-English bilingual evaluation dataset. This new dataset is intended to be published.

Key Insights and Ideas

  • Character-level processing is crucial for low-resource Korean MT: The paper strongly suggests that moving beyond word-level tokenization is essential for handling the grammatical nuances between North and South Korean.
  • Phoneme decomposition effectively addresses initial sound rule differences: By breaking down words into their phonetic building blocks, the model becomes more robust to variations in pronunciation and spelling that arise from the ISR.
  • Combining character tokenization and phoneme decomposition offers the best of both worlds: The "character (phoneme BPE) model" achieved the highest performance, indicating that a multi-layered approach to preprocessing is most effective in mitigating grammatical differences.
  • Zero-shot learning with similar data is viable: The success of this approach demonstrates that even with a lack of direct North Korean parallel data, leveraging closely related languages with appropriate preprocessing can yield significant improvements.

Key Results

The paper reports significant improvements in BLEU scores for North Korean to English translation using their proposed methods. Specifically:

  • The char (phonBPE) model achieved the highest BLEU scores on both development and test sets.
  • For North Korean to English translation, the char (phonBPE) model showed an improvement of +1.01 points compared to the word (charBPE) model.
  • Human evaluation also favored the char (phonBPE) model, showing an improvement of +0.11 points in fluency and +0.02 points in adequacy compared to the word (charBPE) model.
  • Analysis on subsets of data with specific grammatical differences (WS and ISR) indicated that the char (phonBPE) model effectively absorbed these differences, with minimal BLEU score discrepancies between South and North Korean evaluations.

Limitations and Future Work

  • Word Meaning Differences: The paper acknowledges that while grammatical differences are addressed, differences in word meanings (e.g., "낙지" meaning "squid" in South Korean and "octopus" in North Korean) remain a challenge.
  • Contextual Understanding: The current approach might not fully capture nuances arising from words with the same pronunciation and notation but different meanings.
  • Future Work: The authors plan to create an evaluation dataset that accounts for word meaning differences, potentially using North Korean news articles. They also aim to develop translation methods that incorporate language models with stronger contextual understanding, such as BERT, to address these remaining challenges.

Summary updated

BibTeX
@inproceedings{kim2020zero,
  author = {Hwichan Kim and Tosho Hirasawa and Mamoru Komachi},
  title = {{Zero-shot North Korean to English Neural Machine Translation by Character Tokenization and Phoneme Decomposition}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure