A Korean grapheme-to-phoneme conversion system using selection procedure for exceptions

Sunhee Kim, Ju-Eun Ahn, Soon-Hyob Kim, Yang-Hee LeeInterspeech 2004

TL;DR

This paper presents a novel Korean grapheme-to-phoneme (GTP) conversion system that significantly improves accuracy by focusing on exceptions. By developing a two-step process to build an exceptions dictionary from both general dictionaries and large text corpora, they achieve a perfect 100% performance level on a challenging Korean speech recognition dataset.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) and Text-To-Speech (TTS) systems rely heavily on converting written text (graphemes) into spoken sounds (phonemes). This process, known as Grapheme-to-Phoneme (GTP) conversion, typically uses a combination of general pronunciation rules and a dictionary of exceptions. For languages like Korean, which have dynamic vocabularies influenced by cultural and social factors, effectively handling these exceptions is crucial for system performance. Existing methods often struggle to keep pace with new word coinages and nuanced pronunciations, leading to errors in ASR and TTS. This paper addresses the challenge of building a comprehensive and accurate exceptions dictionary for Korean GTP systems that can adapt to the evolving nature of the language.

Methods

The authors propose a hybrid GTP system that integrates morphological analysis, regular phonological rules, and a carefully constructed exceptions dictionary. The core innovation lies in their two-step approach to building this exceptions dictionary:

  1. General Dictionary-Based Compilation: They first extract words exhibiting exceptional pronunciations from a large general Korean dictionary. This involves identifying words within specific phonetic contexts known to trigger irregular alternations. A manual review is then performed to filter these into a basic exceptions dictionary.
  2. Text Corpus-Based Adaptation: Recognizing that general dictionaries don't capture all real-world vocabulary, they introduce a method to adapt the exceptions dictionary using an added text corpus (e.g., from ASR/TTS training data). This involves extracting "Eojols" (Korean word units) from the corpus that fall into exceptional phonetic contexts. These are then compared against the existing exceptions dictionary, and any newly identified exceptions are added. This process effectively "learns" new exceptions from the target domain.

The overall GTP system architecture first performs morphological analysis, then applies rules for exceptions, and finally uses general phonological rules.

Datasets

  • General Korean Dictionary: A dictionary with approximately 50,000 high-frequency entries was used for the initial compilation of the exceptions dictionary.
  • ETRI Speech Recognition Text Corpus: This corpus, consisting of 53,750 sentences (740,497 Eojols), was used for both training (adapting the exceptions dictionary) and evaluation of the GTP system. After removing repeated words, 189,481 unique Eojols were used for experiments.

Key Insights and Ideas

  • The Dynamic Nature of Language: The paper strongly emphasizes that language is not static. New words and pronunciations emerge constantly, making a static exceptions dictionary insufficient.
  • Exceptions are Key to Performance: For Korean GTP, the accuracy of the system hinges on the quality and comprehensiveness of its exceptions dictionary, more so than the regular rules themselves.
  • Corpus-Driven Adaptation is Crucial: Leveraging large text corpora is essential for capturing the real-world usage and evolving exceptions of a language, complementing traditional dictionary-based approaches.
  • A Two-Step Approach Yields Superior Results: Combining a general dictionary-based approach with corpus-driven adaptation provides a robust method for building a highly accurate exceptions dictionary.

Key Results

The paper reports a remarkable 100% performance level for their proposed Korean grapheme-to-phoneme conversion system when using the extended exceptions dictionary derived from the ETRI text corpus. This was achieved after an initial experiment with a basic exceptions dictionary (compiled from a general dictionary) which showed good performance but not perfection. The extended dictionary, incorporating exceptions learned from the text corpus, proved to be the key to achieving flawless conversion.

Limitations and Future Work

  • Limitations: The paper focuses specifically on Korean. The proposed method for exception extraction and compilation might require adaptation for languages with different linguistic structures. While the system achieved 100% accuracy on the specific ETRI corpus, its performance on other, potentially more diverse or noisy, Korean text corpora is not explicitly discussed.
  • Future Work: The authors suggest that their study will contribute to the performance improvement of ASR and TTS systems in Korean. Further research could explore automating the manual review process for exceptions or investigating more sophisticated methods for identifying and categorizing phonetic contexts.

Summary updated

BibTeX
@inproceedings{kim2004korean,
  author = {Sunhee Kim and Ju-Eun Ahn and Soon-Hyob Kim and Yang-Hee Lee},
  title = {{A Korean grapheme-to-phoneme conversion system using selection procedure for exceptions}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure