A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition

Jan Stadermann, Gerhard RigollInterspeech 2004

TL;DR

This paper introduces a novel hybrid approach to automatic speech recognition (ASR) by combining Support Vector Machines (SVMs) with Hidden Markov Models (HMMs). By leveraging SVMs for their discriminative power and HMMs for their temporal modeling capabilities, the authors achieve promising improvements in word error rates, particularly in noisy conditions.

What Problem Does This Paper Address?

Traditional ASR systems often rely on acoustic models based on Hidden Markov Models (HMMs) with Gaussian probability densities. While effective, these models can struggle with the complexities of real-world acoustic environments, especially noise. Neural Networks (NNs) have been integrated into HMM frameworks to improve discriminative capabilities, leading to approaches like tied-posteriors where NNs estimate phoneme posterior probabilities. However, Support Vector Machines (SVMs), another powerful discriminative classifier, have not been as extensively integrated into this tied-posterior framework for ASR. This paper aims to bridge this gap by proposing a hybrid SVM/HMM system that utilizes SVMs to estimate class posterior probabilities, which are then fed into the HMM framework, offering a potentially more robust and discriminative acoustic model.

Methods

The core of this paper is the fusion of SVMs and HMMs. The authors adapt the tied-posterior framework, typically used with NNs, to incorporate SVMs. Instead of NNs, SVMs are trained to calculate class posterior probabilities for each acoustic unit. To handle the temporal nature of speech, these SVM outputs are transformed into probabilities using a sigmoid function. These probabilities are then used to compute state-dependent likelihoods within the HMM framework. This approach avoids the need for segment-level data sampling often required in previous SVM-HMM systems and allows for flexible HMM topologies. The SVMs themselves are trained using a formulation that accounts for non-linearly separable problems by employing kernel functions, specifically the Radial Basis Function (RBF) kernel, to map feature spaces.

Datasets

  • AURORA2 Database: This database, containing spoken digits and digit chains from the TI digits database with various noise types and signal-to-noise ratios (SNRs), is used for evaluation. The experiments specifically utilize the clean-condition training set for training and evaluate performance on the AURORA2 test sets (Test A, Test B, and Test C).

Key Insights and Ideas

  • SVMs as Discriminative Probability Estimators: The paper demonstrates that SVMs can be effectively used to estimate class posterior probabilities, a role traditionally filled by NNs in hybrid systems. This opens up new avenues for leveraging SVMs' discriminative power in ASR.
  • Tied-Posteriors for SVMs: Adapting the tied-posterior concept from NN/HMM systems to SVM/HMM systems allows for independent optimization of HMM topology and input context, leading to a more modular and flexible ASR architecture.
  • Improved Robustness in Noise: The hybrid SVM/HMM system shows promising improvements, especially when dealing with speech distorted by background speech or unknown noise types, suggesting enhanced robustness compared to baseline Gaussian HMMs.
  • Contextual Information for SVMs: Incorporating context frames into the SVM input vector significantly improves performance, highlighting the importance of temporal context for SVM-based acoustic modeling.

Key Results

The paper reports promising results on the AURORA2 task. A hybrid SVM/HMM system trained with 23 half-word SVMs and a substantial amount of training data achieved a significant improvement in word error rate compared to a baseline Gaussian HMM system. For instance, on Test A, the best SVM result showed a relative deviation of -80% compared to the baseline. The system also demonstrated better performance on tests with unknown noise types (Test B) compared to known noise types (Test A), indicating its potential for handling more varied acoustic conditions.

Limitations and Future Work

The authors acknowledge that training one SVM for each acoustical unit (e.g., phoneme) would lead to an impractically large number of SVMs. Therefore, they explored using SVMs for halves of word models. Future work suggested includes integrating SVMs into a NN/HMM tied-posterior system, potentially combining the strengths of both NNs and SVMs for even greater performance gains. They also note that while the system shows improvement, the quality of SVM systems is dependent on noise characteristics, and further research into coping with varying noise types is warranted.

Summary updated

BibTeX
@inproceedings{stadermann2004hybrid,
  author = {Jan Stadermann and Gerhard Rigoll},
  title = {{A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure