Advances in word based dialect/accent classification

Rongqing Huang, John H. L. HansenInterspeech 2005

TL;DR

This paper introduces a "Context Adapted Training" (CAT) algorithm to improve dialect/accent classification for small datasets. By adapting universal phoneme models to dialect-specific word Hidden Markov Models (HMMs), they achieve significant error reductions compared to baseline methods, making dialect classification more robust even with limited training data.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems are often affected by speaker accent or dialect. While accent classification is valuable for improving ASR performance (e.g., through better pronunciation modeling), training robust models for each dialect is challenging, especially when dealing with limited training data. Traditional approaches like Large Vocabulary Continuous Speech Recognition (LVCSR) based systems perform well on large corpora but struggle with data scarcity. The authors previously proposed a "Word based Dialect Classification" (WDC) system that breaks down utterance-level classification into word-level decisions, outperforming LVCSR on large datasets. However, for small training sets, creating accurate HMMs for each word and dialect becomes difficult. This paper addresses this gap by proposing a method to adapt existing models to create more robust dialect-specific word models from limited data.

Methods

The core contribution is the Context Adapted Training (CAT) algorithm. This method aims to adapt universal phoneme Gaussian Mixture Models (GMMs) to create dialect-dependent word HMMs. The process involves:

  1. Phoneme GMM Training: Universal gender-dependent or gender-independent phoneme GMMs are trained using all available dialect data.
  2. Word HMM Initialization: For each word and dialect, an HMM is initialized. The phoneme sequence of a word is obtained either from a pronunciation dictionary (like CMUdict) or a phoneme recognizer. The HMM structure can be left-to-right with or without a one-state-skip capability to handle phoneme deletion.
  3. Viterbi Forced Alignment: This is used to align audio frames to the phoneme sequences and determine state/mixture assignments for training samples.
  4. Parameter Adaptation: The GMM parameters (mean and covariance) of the phoneme models are adapted using a Maximum Likelihood Linear Regression (MLLR) inspired approach. The transition probabilities of the word HMMs are then updated based on the alignment results.
  5. Iteration: Steps 3 and 4 are iterated until convergence or a predefined stopping criterion is met.

The paper describes three variations of CAT: CAT1-a (no-skip, dictionary-based), CAT1-b (no-skip, phoneme recognizer-based), and CAT2 (one-state-skip, dictionary-based).

Datasets

  • IViE corpus: This is the primary dataset used for evaluation. It contains read and spontaneous speech from 8 British English dialects (Belfast, Bradford, Cambridge, Cardiff, Leeds, Liverpool, London, Newcastle). The paper specifically uses read speech for training (64 speakers, 5 hours) and testing word classification, and spontaneous speech for utterance-level dialect classification (32 speakers, 86 minutes). The IViE corpus is characterized by limited training data per dialect (average of less than 40 minutes) and per word (average of 10 training samples).
  • CMU Pronunciation Dictionary: Used to obtain phoneme sequences for words.
  • Wall Street Journal (WSJ) corpus: Mentioned as a benchmark for LVCSR performance in previous work.
  • NATO N4 corpus: Also mentioned as a benchmark in previous work.

Key Insights and Ideas

  • Word-level decisions are powerful: The WDC framework's success on large corpora highlights the effectiveness of breaking down utterance-level classification into multiple word-level decisions.
  • Data scarcity is a major hurdle: The paper strongly emphasizes that training robust HMMs for each word and dialect is extremely difficult with limited data, motivating the need for adaptation techniques.
  • Phoneme models as a universal base: Leveraging universal phoneme models trained on all available data provides a strong starting point for adaptation, as phonemes are more fundamental units than words.
  • MLLR-inspired adaptation is effective: The adaptation of GMM parameters using an MLLR-like approach, combined with Viterbi alignment, proves crucial for creating dialect-specific word models.
  • Handling phoneme variations: The inclusion of a one-state-skip structure (CAT2) acknowledges that phoneme deletion is common and can be important for accurate modeling, though it didn't significantly outperform simpler models in this specific UK-centric dialect set.

Key Results

  • Word Classification: All three CAT methods significantly outperform the baseline WDC training algorithm on words with 3 or more phonemes. The three CAT structures achieved similar performance for word classification.
  • Utterance Classification:
    • The "WDC+CAT" system (using CAT1-a for word models) achieved a 20.2% relative error reduction on utterance dialect classification compared to the basic WDC system.
  • The relative error reduction from the baseline LVCSR system to the "WDC+CAT" system was 35.5%.
  • Specifically, the utterance classification error rates were: LVCSR (32.4%), WDC (26.2%), and WDC+CAT (20.9%).

Limitations and Future Work

  • Dialect Similarity: The paper notes that since all 8 dialects are from the UK, the differences might be less pronounced than if the dialects were more diverse (e.g., including US English). This might explain why CAT1-b and CAT2 did not significantly outperform CAT1-a.
  • Future Work: The authors suggest that for more diverse dialect sets, the CAT1-b and CAT2 structures might show greater advantages. Further exploration into more advanced adaptation techniques could also be beneficial. The paper focuses on English dialects, implying potential for extension to other languages.

Summary updated

BibTeX
@inproceedings{huang2005advances,
  author = {Rongqing Huang and John H. L. Hansen},
  title = {{Advances in word based dialect/accent classification}},
  booktitle = {Interspeech},
  year = {2005},
}
Figure