Noise compensation for speech recognition using subspace Gaussian mixture models

Mohamed Bouallegue, Mickael Rouvier, Driss Matrouf, Georges LinarèsInterspeech 2012

TL;DR

This paper introduces a novel method for compensating for additive noise in speech recognition systems by modeling noise as a low-dimensional subspace within the acoustic feature space. By estimating and subtracting this noise component, the authors achieve a significant reduction in Word Error Rate (WER) on noisy real-world and artificially corrupted speech data.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems are notoriously susceptible to performance degradation in the presence of background noise. This is primarily due to the mismatch between acoustic models trained on clean speech and the noisy features encountered during testing. Existing approaches generally fall into two categories: adapting acoustic models to the noisy environment or denoising the acoustic features directly. While effective, these methods often require extensive noise modeling or can be computationally intensive. This paper tackles the challenge of robustly compensating for additive noise by proposing a method that efficiently estimates and removes the noise component from acoustic features, thereby improving ASR performance without requiring extensive adaptation of the clean speech models.

Methods

The core of this paper's contribution lies in its application of the Subspace Gaussian Mixture Models (SGMM) paradigm for noise compensation. The authors model the acoustic space using a Universal Background Model (UBM) composed of Gaussian Mixture Models (GMMs). Each phoneme is then represented by a GMM derived from the UBM. A key concept is the "supervector," which is formed by concatenating the means of these GMMs, resulting in a very high-dimensional representation.

The proposed SGMM approach models the additive noise as a component residing within a low-dimensional subspace of this supervector space. This is mathematically expressed as: $m_{ph,n} = m_{ph} + D y_{ph} + U x_n$, where $m_{ph,n}$ is the noisy supervector, $m_{ph}$ is the clean phoneme supervector, $D y_{ph}$ represents the linguistic component (modeled as Gaussian noise), and $U x_n$ is the additive noise component. The matrix $U$ defines the noise subspace, and $x_n$ are the "noise factors" within this subspace.

The estimation of the noise component ($U$) is performed iteratively using an Expectation-Maximization (EM) algorithm. This process involves estimating the noise factors ($x_n$) and linguistic components ($y_{ph}$), and then updating the noise subspace matrix ($U$). Crucially, the authors assume that the training data is clean, and the noise estimation and subtraction are applied during the testing phase. For testing, a single noise subspace matrix ($U$) is estimated and used to normalize all utterances in the test corpus.

Datasets

The paper utilizes data from the LIA broadcast news ASR system, SPEERAL [12]. For evaluation, they used:

  • Real-world noisy data: Recorded during several conferences in degraded acoustic conditions, characterized by background noise and poorly placed microphones. This corpus represents a challenging, real-world scenario.
  • Artificially noisy data: This data was created by adding noise to clean speech at various Signal-to-Noise Ratios (SNRs) of 8dB, 1dB, and -8dB. This allowed for controlled testing of the noise estimation and compensation under different noise levels.

Key Insights and Ideas

  • Noise as a Low-Dimensional Subspace: The fundamental insight is that additive noise in the cepstral domain can be effectively represented and modeled within a significantly lower-dimensional subspace compared to the full supervector space. This dimensionality reduction is key to efficient noise estimation.
  • Decoupling Noise and Linguistic Information: The SGMM framework allows for a conceptual separation of the noise component from the linguistic content of the speech signal, enabling targeted removal of the noise.
  • Single Noise Model for Diverse Conditions: A significant advantage highlighted is the ability to estimate a single noise subspace matrix ($U$) that can be applied to normalize various noisy recordings, even those from different environments, suggesting a degree of generality.
  • Efficiency of the Approach: The noise compensation procedure is computationally efficient, with the normalization taking approximately 10 minutes per hour of audio, making it suitable for real-time applications.

Key Results

  • Real-world noisy data: The proposed approach achieved a relative Word Error Rate (WER) reduction of 15% on data recorded in noisy environments.
  • Artificially noisy data: On artificially noised data, the method demonstrated further improvements. Normalizing with a matrix estimated at an SNR close to the test data's SNR (e.g., using UN3 for -8dB SNR data) yielded the largest gains. For instance, the "Total (7h40)" for artificially noised data showed a reduction in WER from 45.90% (unnormalized) to 42.06% when normalized with UN3.

Limitations and Future Work

  • Additive Noise Assumption: The method is primarily designed for additive noise in the time domain. Its effectiveness might be limited in scenarios with non-additive noise or other types of distortions.
  • Training Data Cleanliness: The assumption that training data is clean is crucial. If the training data itself contains significant noise, the learned noise subspace might be inaccurate.
  • Future Work: The authors suggest that this approach could be extended to model other types of noise or distortions and could be integrated into more complex ASR systems. The efficiency of the method also opens doors for real-time applications in degraded acoustic conditions.

Summary updated

BibTeX
@inproceedings{bouallegue2012noise,
  author = {Mohamed Bouallegue and Mickael Rouvier and Driss Matrouf and Georges Linarès},
  title = {{Noise compensation for speech recognition using subspace Gaussian mixture models}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure