Effects of Korean learners' consonant cluster reduction strategies on English speech recognition performance

Hyejin Hong, Jina Kim, Minhwa ChungInterspeech 2010

TL;DR

This paper investigates how Korean learners of English simplify complex consonant clusters, a common challenge in second language acquisition. By modeling these reduction strategies (vowel epenthesis and consonant deletion) and integrating them into speech recognition systems, the authors demonstrate that vowel epenthesis significantly improves recognition accuracy for non-native speech, while consonant deletion degrades it.

What Problem Does This Paper Address?

Recognizing non-native speech is a significant hurdle for current speech recognition systems, leading to substantial performance degradation. This is particularly true for learners of English whose native language has different phonotactic rules. Korean speakers, for instance, struggle with English consonant clusters because Korean syllable structures are much simpler, allowing only single consonants in onsets and codas. This paper addresses the gap in understanding how these L2 learners adapt their pronunciation to simplify these challenging clusters and how these adaptation strategies impact the performance of automatic speech recognition (ASR) systems designed to understand them.

Methods

The authors propose two models to capture Korean learners' consonant cluster reduction strategies:

  • Vowel Epenthesis Model: This model incorporates the insertion of epenthetic vowels into consonant clusters. This strategy is based on phonetic and phonological analysis of Korean learners' speech, where vowels are added to break up difficult consonant sequences. The model modifies the pronunciation dictionary by adding these epenthetic vowels.
  • Consonant Deletion Model: This model reflects the strategy of deleting consonants within a cluster. This deletion is guided by the Sonority Hierarchy (SH) and Sonority Sequencing Principle (SSP), which predict which consonants are more likely to be dropped to create a more sonority-smooth syllable. The pronunciation dictionary is also modified to include these deleted consonants.

Both models are then integrated into an ASR system, and their performance is evaluated against a baseline system.

Datasets

  • Training Data:
    • Native English: WSJ corpus (WSJ 0 & WSJ 1) for training the baseline and native speech models.
  • Non-native English: Korean-Spoken English Corpus (K-SEC) for training non-native speech models. This corpus is further divided into adult and elementary school learner sets.
  • Testing Data:
    • Native English: WSJ 0 (si et 05).
  • Non-native English: K-SEC (adult) and K-SEC (child) sets.

The paper also mentions the use of the CMU Pronouncing Dictionary as a baseline for pronunciation.

Key Insights and Ideas

  • L2 learners actively simplify: Korean learners don't just produce errors; they employ systematic strategies to simplify English consonant clusters that are difficult for them.
  • Vowel epenthesis is a "safer" strategy: The vowel epenthesis strategy, which adds vowels to break up clusters, leads to improved ASR performance. This suggests it's a more robust adaptation that the ASR system can better handle.
  • Consonant deletion is detrimental: Conversely, the consonant deletion strategy, while simplifying pronunciation for the learner, significantly degrades ASR performance. This indicates that deleting sounds can lead to more ambiguity and misinterpretation by the system.
  • Linguistic theory aligns with ASR outcomes: The findings strongly support linguistic theories suggesting that L2 learners prefer vowel epenthesis over consonant deletion when simplifying clusters. This bridges the gap between linguistic analysis and practical ASR applications.

Key Results

  • Vowel Epenthesis Model:
    • For Korean adult learners, it resulted in an absolute Word Error Rate (WER) reduction of 0.26% (1.60% relative improvement).
  • For Korean elementary school learners, it achieved an absolute WER reduction of 0.43% (1.80% relative improvement).
  • Consonant Deletion Model:
    • Degraded performance for both adult and elementary school learners, increasing WER by 0.62% and 1.38% respectively.
  • Native Speech: Both proposed models degraded performance for native English speakers, as expected, since they are designed to model non-native speech patterns.

Limitations and Future Work

  • Limitations: The paper acknowledges that the consonant deletion model's performance degradation is consistent with linguistic studies, but the exact reasons for this degradation in ASR are not fully explored. The study focuses on specific strategies (vowel epenthesis and consonant deletion) and may not cover all possible reduction techniques.
  • Future Work: The authors plan to investigate other coping strategies used by Korean learners and examine their effects on English speech recognition to further improve performance. They also aim to develop a non-native speech recognizer specifically for a Computer Assisted Language Learning (CALL) system for Korean learners.

Summary updated

BibTeX
@inproceedings{hong2010effects,
  author = {Hyejin Hong and Jina Kim and Minhwa Chung},
  title = {{Effects of Korean learners' consonant cluster reduction strategies on English speech recognition performance}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure