Unsupervised training scheme with non-stereo data for empirical feature vector compensation

Luis Buera, Antonio Miguel, Alfonso Ortega, Eduardo Lleida, Richard M. SternInterspeech 2009 · DOI

Introduction

Automatic Speech Recognition (ASR) systems often suffer from performance degradation in noisy environments, such as inside a moving vehicle. To mitigate this, feature vector normalization techniques like Multi-Environment Model-based LInear Normalization (MEMLIN) are employed to map noisy speech signals back to a clean state. Traditionally, these models require "stereo data"—simultaneous recordings of the same speech in both clean and noisy conditions—to learn the transformation. However, collecting such data is often impractical or impossible in real-world scenarios. This paper proposes an unsupervised training scheme that utilizes non-stereo data, combining Maximum Likelihood (ML) estimation with Vector Taylor Series (VTS) to bridge the gap between clean and noisy acoustic spaces.

Methodology

The core of the proposed approach is to adapt the MEMLIN framework for non-stereo training. Standard MEMLIN uses Gaussian Mixture Models (GMMs) to represent acoustic spaces and learns a bias vector to compensate for noise. The authors introduce two key components to eliminate the need for stereo pairs:

  1. Vector Taylor Series (VTS) Initialization: VTS provides a mathematical model of how additive noise and convolutional distortion affect speech. It is used here to generate an initial "guess" of the relationship between clean and noisy Gaussians.
  2. Expectation-Maximization (EM): An iterative algorithm is then used to refine this relationship based on the Maximum Likelihood criterion, effectively estimating the hidden mapping between independent sets of clean and noisy data.

This methodology is also applied to MEMLIN-CPM, which utilizes a cross-probability model to capture more complex, many-to-many relationships between acoustic states, further enhancing robustness.

Experimental Setup and Baseline

The approach was evaluated using the Spanish SpeechDat Car database, which includes seven distinct driving conditions (e.g., town traffic, high-speed driving). Clean data was captured via a close-talk (CLK) microphone, while noisy data came from a ceiling-mounted hands-free (HF) microphone.

Baseline Word Error Rates (AWER) across different training and testing conditions in the SpeechDat Car database.

As shown in the baseline results, there is a significant performance drop when a model trained on clean data (CLK) is tested in a noisy environment (HF), with the Average Word Error Rate (AWER) jumping from 1.0% to 8.5%.

Performance Evaluation

The researchers measured performance using Average Improvement (AIMP), which quantifies error reduction relative to the clean baseline. The results demonstrate that the non-stereo (NS) versions of the models are highly effective.

Comparison of AIMP (%) across different methods as a function of the number of noisy model Gaussians.

Key findings from the evaluation include:

  • NS-MEMLIN-CPM: Reached a 61.9% AIMP, reducing the AWER from 8.5% to approximately 3.8%.
  • Comparison to VTS: The non-stereo MEMLIN-CPM outperformed standard VTS-based normalization (70.8% AIMP) when considering the trade-off between accuracy and the computational cost required during the decoding phase.
  • The Stereo Gap: While stereo-trained models still provide the highest performance (e.g., MEMLIN-CPM at 81.7% AIMP), the non-stereo approach offers a practical and competitive alternative for real-world applications where stereo data is unavailable.

Conclusion

This work successfully demonstrates that high-performance feature normalization models can be trained without expensive stereo recordings. By using VTS for initialization and EM for unsupervised refinement, the proposed scheme significantly improves ASR accuracy in noisy car environments. Future research will focus on more complex signal models to further close the performance gap between stereo and non-stereo training regimes.

Summary updated

BibTeX
@inproceedings{buera2009unsupervised,
  author = {Luis Buera and Antonio Miguel and Alfonso Ortega and Eduardo Lleida and Richard M. Stern},
  title = {{Unsupervised training scheme with non-stereo data for empirical feature vector compensation}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-359},
}
Figure