TL;DR
This paper tackles the challenge of Korean Large-Vocabulary Continuous Speech Recognition (LVCSR) by proposing a novel two-stage approach. First, it performs standard speech recognition on phonemic syllable "surface forms" to mitigate acoustic confusability. Second, it uses a noisy-channel model to translate these phonemic forms into the desired orthographic units (eumjeol or eojeol), achieving significant improvements in accuracy.
What Problem Does This Paper Address?
Developing accurate Large-Vocabulary Continuous Speech Recognition (LVCSR) systems for agglutinative and highly inflective languages like Korean presents unique challenges. Unlike languages with clear word boundaries (like English), Korean's orthographic units, "eojeol" (word-phrases), can be formed by combining many morphemes, leading to extremely high perplexity and out-of-vocabulary (OOV) rates if used as recognition units. Conversely, using "eumjeol" (orthographic syllables) results in small dictionaries and low OOV rates but suffers from high acoustic confusability due to complex phonological phenomena and coarticulation effects. Existing approaches often require extensive linguistic knowledge or complex data-driven methods. This paper aims to overcome these issues by decoupling the acoustic recognition from the orthographic representation.
Methods
The proposed approach is a two-stage serial architecture:
- Phonemic Syllable Recognition (Surface Form): This first stage uses a standard Hidden Markov Model (HMM)-based speech recognition system. The key innovation here is using phonemic syllable units representing the actual pronunciation (surface forms) as the recognition unit. This ensures a one-to-one mapping between a phonemic syllable symbol and its pronunciation, keeping the lexicon small and avoiding acoustic confusability. This part is essentially a standard LVCSR system but operating on a different unit.
- Noisy-Channel Translation: The second stage takes the recognized sequence of phonemic syllables (considered a "noisy" string) and transforms it into the desired orthographic Korean sequence (eumjeol or eojeol, the "clean" string). This is achieved using a noisy-channel model, a technique borrowed from statistical machine translation (SMT). The model learns the probability of generating an orthographic unit from a sequence of phonemic syllables. This process does not require explicit linguistic knowledge, relying solely on annotated text data.
Datasets
The experiments were conducted using the large-vocabulary continuous Korean speech database developed by the Speech Information Technology and Industry Promotion Center (SiTEC) [20]. Three types of corpora were used:
- Phonetically-balanced sentences (Sent01): ~20,000 sentences, recorded by 200 speakers.
- Dictation application sentences (Dict01 and Dict02): ~40,000 sentences each, recorded by 400 speakers each.
The data was split into training and test sets, with the last two prompt sets from each corpus designated for testing.
Key Insights and Ideas
- Decoupling Acoustic and Orthographic Units: The core idea is to separate the acoustic modeling from the final orthographic representation. By recognizing "surface forms" first, the system bypasses the direct acoustic confusability issues associated with orthographic units like eumjeol.
- Noisy-Channel Model for Translation: Leveraging the noisy-channel framework from SMT is a clever way to handle the transformation from phonemic to orthographic representations without needing deep linguistic analysis. This makes the approach more generalizable.
- Efficiency of Phonemic Syllables: Using phonemic syllables as intermediate units significantly reduces the lexicon size and OOV rates compared to using eojeol, while avoiding the acoustic confusability of eumjeol.
- Data-Driven Approach: The entire process relies on annotated text, making it adaptable to other agglutinative languages without requiring extensive manual linguistic feature engineering.
Key Results
The proposed system achieved significant improvements over the baseline LVCSR system:
- Baseline System (Orthographic Syllable - Eumjeol): Achieved accuracies of 70.13% (Sent01), 67.96% (Dict01), and 57.39% (Dict02).
- Proposed System (Noisy-Channel Approach):
- Phonemic Syllable Accuracy (intermediate step): Achieved very high accuracies, e.g., 88.67% on Sent01.
- Final Orthographic Eumjeol Accuracy: Achieved 91.21% (on Sent01 test set).
- Final Orthographic Eojeol Accuracy: Achieved 71.30% (on Sent01 test set).
This represents a substantial 26.67% absolute improvement in orthographic syllable (eumjeol) accuracy compared to the baseline.
Limitations and Future Work
- Limitations: The paper acknowledges that the phonological changes in Korean are complex, and the proposed system might not capture all nuances. The performance on eojeol accuracy (71.30%) is still lower than eumjeol accuracy (91.21%), indicating room for improvement in the translation stage.
- Future Work: The authors suggest that this approach, requiring only annotated texts, is applicable to other agglutinative languages. Further research could explore more sophisticated noisy-channel models or integrate linguistic knowledge more effectively if available.
Summary updated