Noise compensation for subspace Gaussian mixture models

Liang Lu, K. K. Chin, Arnab Ghoshal, Steve RenalsInterspeech 2012

TL;DR

This paper introduces a noise compensation technique called Joint Uncertainty Decoding (JUD) for Subspace Gaussian Mixture Models (SGMMs), a more complex but powerful type of acoustic model for speech recognition. By adapting JUD to SGMMs, the authors achieve significant improvements in speech recognition accuracy in noisy conditions, outperforming traditional GMM-based methods.

What Problem Does This Paper Address?

Speech recognition systems struggle in noisy environments because the acoustic signals are distorted, making it difficult for the models to accurately identify spoken words. Traditional methods like Vector Taylor Series (VTS) attempt to compensate for this noise by adapting the acoustic models themselves. However, modern acoustic models like Subspace Gaussian Mixture Models (SGMMs) are significantly larger and more complex than older Gaussian Mixture Models (GMMs). This complexity makes VTS computationally prohibitive, as it requires adapting every single Gaussian component within the model. The paper aims to find a more efficient and effective way to perform noise compensation specifically for these advanced SGMM models.

Methods

The core of this paper is the application of Joint Uncertainty Decoding (JUD) to Subspace Gaussian Mixture Models (SGMMs).

  • JUD (Joint Uncertainty Decoding): This is a model-based noise compensation technique that treats the relationship between clean and noisy speech observations as a probabilistic model. Instead of directly adapting each Gaussian component (like VTS), JUD groups Gaussians into "regression classes." This allows a single noise compensation transformation to be applied to all Gaussians within a class, significantly reducing computational cost.
  • SGMM (Subspace Gaussian Mixture Model): SGMMs are an advancement over traditional GMMs. They use a low-dimensional subspace to represent Gaussian means and weights, allowing for a much larger number of effective Gaussians without a proportional increase in the number of parameters to estimate. This makes them more powerful but also more challenging for traditional adaptation techniques.
  • UBM as Regression Model: A key innovation in this paper is using the Universal Background Model (UBM) – a standard component in GMM/SGMM systems – as the basis for the regression classes in JUD. This avoids the computationally expensive step of clustering the millions of SGMM surface Gaussians, leveraging the UBM's inherent acoustic partitioning.
  • Noise Model Estimation: The paper uses a gradient-based approach to estimate the noise model parameters, which is a standard technique for VTS and JUD.
  • Phase Factor: The authors also investigate the impact of a "phase factor" (denoted by 'α') in the noise compensation process, which can capture correlations between noise and speech.

Datasets

  • Aurora 4 Corpus: This dataset, derived from the Wall Street Journal (WSJO) 5k-word task, was used for evaluation. It includes clean training data and noisy versions created by adding various types of noise at different Signal-to-Noise Ratios (SNRs). The test sets are categorized into A (clean), B (various noises, 5-15dB SNR), C (desk-mounted mic, similar noises), and D (desk-mounted mic, similar noises).

Key Insights and Ideas

  • JUD is Scalable to SGMMs: The paper demonstrates that JUD, a technique previously applied to GMMs, can be effectively adapted to the much larger and more complex SGMM framework.
  • UBM Simplifies SGMM JUD: Using the UBM as the regression model for JUD is a crucial insight. It bypasses the need for explicit clustering of SGMM components, making the approach computationally feasible and efficient.
  • Phase Factor Matters: The investigation into the phase factor (α) reveals its significant impact on noise compensation accuracy, particularly for JUD with SGMMs. Tuning this factor led to substantial improvements, suggesting that capturing phase-related correlations is important.
  • SGMMs Don't Inherently Beat Noise: While SGMMs offer advantages in clean conditions, the paper shows they don't automatically outperform GMMs in highly mismatched noisy conditions without explicit compensation. This highlights the necessity of techniques like JUD.

Key Results

  • Improved Accuracy: The JUD/SGMM system achieved significantly lower Word Error Rates (WERs) on the Aurora 4 corpus compared to baseline clean and MTR (Multi-Condition Training) SGMM systems.
  • Outperforming GMM-based Methods: The JUD/SGMM system, especially when the phase factor was tuned, achieved better results than GMM systems using either VTS or JUD.
  • Specific WERs:
    • For GMM systems (α=0): VTS achieved a best WER of 18.3%, while JUD achieved 21.1%.
  • For SGMM systems (α=0): The baseline MTR SGMM had an average WER of 22.2%. JUD compensation reduced this to 20.3%.
  • With tuned phase factor (α=2.5): The JUD/SGMM system achieved a best WER of 16.8%, outperforming the best VTS/GMM by 0.5%.

Limitations and Future Work

  • Limitations:
    • The paper notes that while JUD/SGMM with a tuned phase factor achieved excellent results, the phase factor was empirically tuned rather than analytically determined.
  • The paper mentions that the VTS linearisation error might be a factor in why tuning the phase factor helps JUD/SGMM.
  • Future Work:
    • Analytical determination of the phase factor.
  • Exploring higher-order VTS approximations for improved accuracy.
  • Investigating extended VTS for better dynamic coefficient estimation.

Summary updated

BibTeX
@inproceedings{lu2012noise,
  author = {Liang Lu and K. K. Chin and Arnab Ghoshal and Steve Renals},
  title = {{Noise compensation for subspace Gaussian mixture models}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure