Target speech GMM-based spectral compensation for noise robust speech recognition

Takahiro Shinozaki, Sadaoki FuruiInterspeech 2009

TL;DR

This paper introduces a novel spectral compensation method (TGSC) that transforms noisy speech spectra before feature extraction, using Gaussian Mixture Models (GMMs) to optimize the transformation parameters. This approach effectively handles both additive and convolutional noise, outperforming spectral subtraction and offering a more robust front-end for speech recognition systems in noisy environments.

What Problem Does This Paper Address?

Speech recognition systems struggle in real-world scenarios due to background noise, which can be either additive (e.g., a car horn) or convolutional (e.g., room acoustics). While traditional methods like spectral subtraction can mitigate additive noise, they often fail with convolutional noise and struggle with non-stationary noise. Existing methods that attempt to address both types of noise often require complex HMM-based state alignment, making them unsuitable for real-time, front-end processing. This paper aims to bridge this gap by proposing a spectral compensation technique that is effective for both noise types, operates independently of the decoder, and can adapt quickly to changing noise conditions.

Methods

The core of the proposed method, Target Speech GMM-based Spectral Compensation (TGSC), lies in transforming the noisy speech spectrum before it's processed into standard features like MFCCs. The transformation is designed to address both additive and convolutional noise, which are modeled as multiplicative and subtractive terms in the spectral domain.

The key innovation is how the parameters of this transformation are optimized. Instead of relying on HMMs and state alignment, TGSC uses a speech GMM (trained on clean speech) to maximize the likelihood of the transformed spectrum. This allows the compensation to be performed as a front-end process, independent of the speech recognition decoder.

The transformation itself is formulated to handle potential negative values after spectral subtraction by incorporating a flooring operation. To enable gradient-based optimization, a continuous approximation of this transformation is used. The optimization process involves estimating parameters for short segments of speech (around 500ms), allowing for rapid adaptation to non-stationary noise. The paper also explores different initialization strategies for the transformation parameters, including using a constant value or an estimated noise vector.

Datasets

The experiments were conducted using the AURORA-2J database. This database is a standard benchmark for noisy speech recognition, particularly for Japanese. The training data consisted of 8440 utterances from 110 speakers, and the test set ("test set C") included 2002 utterances per Signal-to-Noise Ratio (SNR) condition, with seven different SNRs ranging from clean to very noisy. The additive noises used were subway and street noises.

Key Insights and Ideas

  • Spectral Domain Transformation is Key: Operating in the spectral domain before feature extraction offers a more direct way to combat noise effects that get diffused and complicated by subsequent processing steps (like filter banks and log transforms).
  • GMMs for Optimization, Not Decoding: Leveraging GMMs for parameter optimization, rather than HMMs, decouples the noise compensation from the recognition engine, making it a versatile front-end.
  • Adaptability is Crucial: Estimating transformation parameters over short, adaptive segments (around 500ms) allows the system to handle non-stationary noise effectively.
  • Initialization Matters: The choice of initial parameters for the transformation significantly impacts performance, with using an estimated noise vector yielding better results than a constant initialization.
  • Trade-off between Complexity and Performance: While TGSC is computationally more expensive than spectral subtraction due to the optimization process, it offers significant performance gains, especially at lower SNRs.

Key Results

  • Outperforms Spectral Subtraction: TGSC, particularly when initialized with an estimated noise vector, consistently outperformed spectral subtraction across all tested SNR conditions.
  • Significant Improvements: The paper reports statistically significant improvements in word accuracy for TGSC compared to both a baseline (no compensation) and spectral subtraction. For instance, at an SNR of 0 dB, TGSCss achieved 50.0% word accuracy, compared to 41.6% for spectral subtraction.
  • GMM Mixture Count: Improvements were observed with an increasing number of GMM mixtures, with performance largely converging around 400 mixtures.
  • Iteration Count: The optimization process benefited from multiple iterations, with five iterations generally providing the best results, suggesting a potential for over-fitting with more iterations.
  • Combination with CMS: Combining TGSC with Cepstral Mean Subtraction (CMS) further boosted performance, with TGSC initialized with a noise vector and followed by CMS achieving the best overall results.

Limitations and Future Work

  • Computational Cost: TGSC is computationally more expensive than spectral subtraction due to the iterative optimization process. The authors note that their software was not optimized for efficiency, but the cost was largely linear to the number of iterations.
  • Normalization Time Scale: The current normalization time scale is around 500ms. For longer temporal characteristics, combining TGSC with techniques like CMS is suggested.
  • Vocal Tract Length Normalization (VTLN): TGSC does not inherently handle vocal tract length differences between speakers, a capability present in methods like MLLR. The authors suggest exploring combinations with MLLR to leverage its VTLN capabilities.
  • Initialization Strategy: Further investigation into optimal initialization strategies for the transformation parameters is recommended.
  • Exploiting Parallelism: The authors express interest in exploring computationally more expensive variants of TGSC by utilizing parallel processing capabilities like GPGPU for higher noise compensation performance.

Summary updated

BibTeX
@inproceedings{shinozaki2009target,
  author = {Takahiro Shinozaki and Sadaoki Furui},
  title = {{Target speech GMM-based spectral compensation for noise robust speech recognition}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure