Sparse Bayesian factor analysis for stereo-based stochastic mapping

Xiaodong Cui, Mohamed Afify, George Saon, Vaibhava GoelInterspeech 2012

TL;DR

This paper introduces a novel approach to robust speech recognition by using sparse Bayesian factor analysis within a stereo-based stochastic mapping (SSM) framework. By learning a parsimonious representation of underlying acoustic factors, their method significantly improves speech recognition performance in noisy environments compared to standard SSM techniques.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems often struggle with performance degradation in the presence of background noise. Stereo-based Stochastic Mapping (SSM) is a technique that leverages audio from two channels (e.g., a clean and a noisy channel) to learn a mapping that can reconstruct cleaner speech. However, standard SSM, often based on Gaussian Mixture Models (GMMs), can be computationally intensive and may not efficiently capture the underlying generative factors of speech in noisy conditions, especially when dealing with diverse noise types and signal-to-noise ratios (SNRs). This paper aims to address this by developing a more efficient and robust SSM model that can better represent the complex statistical relationships in stereo audio under various noisy conditions.

Methods

The core of this paper is the application of Sparse Bayesian Factor Analysis to the joint channel space of SSM. Here's a breakdown of their approach:

  • Stereo-based Stochastic Mapping (SSM): The foundation is an SSM model that assumes a GMM to represent the joint distribution of clean and noisy speech features. Each Gaussian component in this GMM captures a specific statistical relationship between the two channels.
  • Bayesian Factor Analysis: Instead of directly modeling the covariance of each GMM component, they employ Bayesian factor analysis. This technique assumes that the observed data (joint speech features) can be represented as a linear combination of a smaller set of underlying latent factors, plus some noise.
  • Sparsity-Promoting Prior: The key innovation is the use of a sparsity-promoting prior on the matrix of factor loadings (which maps latent factors to observed features). This prior encourages many of the factor loadings to become zero, effectively selecting only the most important factors for representing the data. This leads to a more parsimonious model and automatically determines the "effective number of factors."
  • Mixture of Sparse Factor Analyzers: The overall model is a "mixture of sparse factor analyzers," where each Gaussian component of the SSM is modeled by a sparse Bayesian factor analyzer.
  • Parameter Estimation: Parameters are estimated iteratively using an Expectation-Maximization (EM) algorithm, interleaved with Type-II maximum likelihood estimation for hyper-parameters. Initialization is done using Probabilistic PCA (PPCA).

Datasets

  • Training Data: 60 hours of continuous clean speech signals.
  • Noise Generation: Noisy speech signals were artificially generated by corrupting the clean speech with 10 types of background noise (including M109, Buccaneer, Leopard, etc.) at SNRs ranging from 25dB to 10dB. The NOISEX-92 dataset was used for noise samples.
  • Test Sets:
    • Set A (Seen Conditions): Noisy speech from DARPA Transtac Nov08 offline test set, corrupted with 5 types of background noise present in the training set, at SNRs of 20dB to 5dB.
  • Set B (Unseen Conditions): Noisy speech from DARPA Transtac Oct09 offline test set, corrupted with 5 types of background noise not present in the training set, at SNRs of 20dB to 5dB.
  • Set C (Real Conditions): Speech signals recorded under real-world humvee-tank noise conditions with SNRs of 5-8dB.

Key Insights and Ideas

  • Parsimony through Sparsity: The core idea is that by forcing sparsity on the factor loadings, the model can automatically identify and use only the most relevant underlying factors, leading to a more efficient and robust representation of noisy speech.
  • Effective Factor Selection: The sparsity prior effectively "switches off" unnecessary factors, reducing model complexity and improving generalization. This is visualized in Figure 1, showing that many factors are indeed unused.
  • Improved Noise Robustness: The sparse Bayesian factor analysis approach demonstrably leads to better performance than standard SSM with full covariance matrices, especially in challenging noisy conditions.
  • Contextual Information Benefits: Incorporating contextual frames (using adjacent noisy speech features) further enhances the performance of the sparse SSM model.

Key Results

The paper reports significant improvements in Word Error Rate (WER) using their proposed sparse Bayesian factor analysis SSM approach.

  • Without Context (SSM, c=0): The sparse MBFA (Mixture of Bayesian Factor Analyzers) achieved a WER of 27.2% on Set A, compared to 31.0% for diagonal covariance SSM and 27.8% for full covariance SSM.
  • With Context (SSM, c=1): The sparse MBFA achieved a WER of 26.8% on Set A, outperforming full covariance SSM by 0.8% absolute. Similar improvements were observed across all test sets (A, B, and C).
  • Figure 1: Illustrates that for both c=0 and c=1, a significant portion of the factors in the dictionary are "switched off" by the sparsity prior, indicating effective model simplification.

Limitations and Future Work

  • Limitations: The paper focuses on multi-conditional training using artificially generated noise. While Set C provides real-world noise, the primary training and evaluation rely on synthesized noise. The computational cost of iterative parameter estimation, though improved by sparsity, can still be a factor.
  • Future Work: The authors suggest exploring more applications of this sparse Bayesian factor analysis technique. They also mention that the effective number of factors might not change significantly even with contextual information, hinting at further investigation into the nature of these underlying factors.

Summary updated

BibTeX
@inproceedings{cui2012sparse,
  author = {Xiaodong Cui and Mohamed Afify and George Saon and Vaibhava Goel},
  title = {{Sparse Bayesian factor analysis for stereo-based stochastic mapping}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure