Unsupervised training of an HMM-based speech recognizer for topic classification

Herbert Gish, Man-hung Siu, Arthur Chan, Bill BelfieldInterspeech 2009

TL;DR

This paper introduces a novel unsupervised training method for HMM-based speech recognizers, eliminating the need for transcribed data. This breakthrough allows for more robust topic classification, especially in domains where labeled speech is scarce, by jointly optimizing acoustic and language models with the transcription itself.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems are crucial for many applications, including spoken topic classification. Traditionally, training these ASR systems requires large amounts of transcribed speech data. This poses a significant challenge for specialized domains, languages, or noisy environments where such data is difficult or impossible to obtain. Using a pre-trained recognizer from a different domain often leads to performance degradation due to mismatches in acoustic conditions, language, or vocabulary. This paper tackles this problem by proposing a method to train ASR models without any transcribed data, making them adaptable to new domains more effectively.

Methods

The core of the paper is an unsupervised training approach for Hidden Markov Models (HMMs). Instead of relying on pre-transcribed data, the method iteratively optimizes both the HMM's parameters (acoustic and language models) and the transcription sequences themselves. This is achieved through a two-step iterative process:

  1. Parameter Optimization: Given a current set of transcription sequences, the HMM parameters are updated using standard supervised training techniques.
  2. Transcription Optimization: Using the updated HMM parameters, the system performs a Viterbi decoding pass to find the most likely transcription sequence for the audio.

This iterative process effectively searches for the optimal parameters and transcription simultaneously.

For initialization, the authors employ a Segmental Gaussian Mixture Model (SGMM) tokenizer. This tokenizer first segments speech based on spectral discontinuities and then models these segments using polynomial trajectories. Clustering these segment models allows for the generation of initial, phone-like units. These units then serve as the initial "labels" for the unsupervised HMM training.

The trained HMM recognizer then acts as a tokenizer for downstream applications like topic classification. Features are extracted from the recognized unit sequences and fed into a Support Vector Machine (SVM) classifier.

Datasets

  • Switchboard Corpus: Used for evaluating the topic classification performance. A subset of 10 target topics was selected.
  • Training Data: 9 hours of non-target conversations were used to train the initial SGMM tokenizer.
  • Test Data: Conversations from the 10 target topics were used for classification.

Key Insights and Ideas

  • Unsupervised Training is Viable for ASR: The paper demonstrates that it's possible to train effective HMM-based speech recognizers without any transcribed data, significantly broadening the applicability of ASR technology.
  • Joint Optimization is Key: The success of the unsupervised approach hinges on simultaneously optimizing both the model parameters and the transcription sequences, allowing the system to learn from its own predictions.
  • SGMM as an Effective Initializer: The SGMM tokenizer provides a robust starting point for the unsupervised HMM training, generating meaningful initial "labels" that capture acoustic units.
  • Domain Adaptation is Crucial: The performance gains observed when using the unsupervised model compared to mismatched supervised models highlight the importance of training data that matches the target domain.

Key Results

The unsupervised HMM recognizer achieved a 12.5% Average Equal Error Rate (EER) on the topic classification task using the Switchboard corpus. This significantly outperformed:

  • A supervised English phoneme recognizer trained with 1 hour of transcribed data (14.7% EER).
  • A state-of-the-art Hungarian phoneme recognizer trained with 10 hours of data (16.7% EER), even though the Hungarian recognizer had a matched channel but a mismatched language.

The unsupervised SGMM-only tokenizer (without HMM training) performed poorly at 30.2% EER, underscoring the benefit of the subsequent unsupervised HMM training.

Limitations and Future Work

  • Phone-like Units: Currently, the HMM is trained to produce phone-like units. The authors suggest exploring the possibility of extending this to "word-like" units for recognition.
  • Single Tokenization: The current approach uses a single tokenization output. Future work could involve generating classification features from multiple tokenizers or using recognition lattices instead of just the 1-best recognition output.
  • Broader Applications: The unsupervised HMM training method is expected to benefit other speech applications, including spoken term detection and speaker identification.

Summary updated

BibTeX
@inproceedings{gish2009unsupervised,
  author = {Herbert Gish and Man-hung Siu and Arthur Chan and Bill Belfield},
  title = {{Unsupervised training of an HMM-based speech recognizer for topic classification}},
  booktitle = {Interspeech},
  year = {2009},
}

Loading related content...

Figure