Transforming features to compensate speech recogniser models for noise

R. C. van Dalen, F. Flego, M. J. F. GalesInterspeech 2009

TL;DR

This paper introduces a new way to make speech recognition systems more robust to noisy environments by transforming features. Instead of just trying to clean up the audio, they adapt the speech recognition models themselves to better handle corrupted speech, achieving better accuracy than existing methods while maintaining speed.

What Problem Does This Paper Address?

Speech recognition systems struggle when background noise interferes with the audio. Traditional approaches either try to "clean" the audio signal before it reaches the recognizer (feature enhancement) or adapt the recognition models to account for the noise. While feature enhancement is fast, it often only provides a point estimate of the clean speech, losing valuable information about uncertainty. Model compensation can be more accurate but is often computationally expensive. This paper tackles the challenge of achieving both accuracy and speed in noise compensation by proposing new methods that transform features in a way that effectively adapts the back-end speech recognition models.

Methods

The paper builds upon the concept of predictive linear transformations (PLTs), which adapt speech recognition models based on statistics predicted from noisy speech. The authors re-analyze PLTs as minimizing the Kullback-Leibler (KL) divergence between the predicted corrupted speech distribution and the adapted model distributions.

Key novel approaches include:

  • Observation-dependent transformations: Unlike previous methods that applied transformations based on the back-end component or noise condition, these new methods adapt the transformation based on the current observation (feature vector). This allows for more context-specific adaptation.
  • Component-independent PCMLLR: This approach estimates a single transformation that is applied to all components of the speech recognition model, simplifying the process.
  • Global PCMLLR: A baseline where a single transform is estimated for each noise condition, independent of the specific observation.
  • Observation-trained PCMLLR: This method estimates a transform for each feature vector by adapting the predicted corrupted speech distribution based on the observation.
  • Hard-decision PCMLLR: A simple scheme that selects a pre-trained transform based on the most likely front-end component.
  • Interpolated PCMLLR: A more sophisticated approach that interpolates between multiple pre-trained transforms, weighted by the front-end component posteriors. This method aims to smoothly adapt to different acoustic regions.

The core idea is to transform the features in a way that effectively transforms the models, rather than trying to reconstruct a clean speech signal.

Datasets

  • AURORA 2: This is the primary dataset used for evaluation. It's a small vocabulary digit string recognition task based on the TIDIGITS database, with artificially added noise. The paper reports results on test set A, averaged over four noise conditions, and notes that other test sets (B and C) showed similar trends.

Key Insights and Ideas

  • KL Divergence as the Objective: The paper clarifies that predictive linear transformations aim to minimize the KL divergence between the predicted corrupted speech and the adapted models, providing a principled theoretical grounding.
  • Feature Transformation as Model Compensation: The key insight is that by carefully transforming the input features, one can effectively achieve model compensation without directly modifying the back-end HMM parameters. This offers a speed advantage.
  • Observation-Dependent Adaptation is Crucial: Adapting transformations based on the specific observation (feature vector) leads to significantly better performance, especially at lower Signal-to-Noise Ratios (SNRs).
  • Interpolation Bridges the Gap: Interpolated PCMLLR, by smoothly blending transforms appropriate for different acoustic regions, achieves the best performance, outperforming even standard MMSE feature enhancement. This highlights the benefit of combining local adaptation with a global framework.
  • MMSE Limitations: The paper shows that standard MMSE feature enhancement, which reconstructs a point estimate of clean speech, performs less well than distribution-based compensation methods at lower SNRs.

Key Results

  • Interpolated PCMLLR Dominates: On the AURORA 2 task, Interpolated PCMLLR achieved the best word error rates across various SNRs, outperforming other PCMLLR variants, global PCMLLR, and standard MMSE. For example, at 0 dB SNR, Interpolated PCMLLR achieved a 10.8% word error rate, compared to 12.7% for MMSE.
  • Speed vs. Accuracy Trade-off: While VTS (Vector Taylor Series) compensation offers high accuracy, it is computationally expensive. The proposed PCMLLR-based methods offer a better trade-off, achieving competitive accuracy with significantly faster processing.
  • Model-MMSE Ineffectiveness: The paper demonstrates that applying MMSE transforms separately to each base class (Model-MMSE) fails to provide meaningful compensation, highlighting the difference in their underlying principles compared to PCMLLR.

Limitations and Future Work

  • Computational Cost of VTS: The paper acknowledges that the most computationally expensive part of the process is often the VTS compensation used to derive the joint distribution. While the proposed methods are faster than VTS model compensation, they still rely on it for initial statistics.
  • "Global PCMLLR" Limitation: The "Global PCMLLR" scheme, while simple, estimates one transform per noise condition and does not adapt to the specific observation, limiting its performance at lower SNRs.
  • Future Work: The paper implicitly suggests that further exploration of interpolation strategies and more sophisticated ways to estimate observation-dependent transforms could lead to even greater improvements. The success of interpolated PCMLLR points towards the benefit of smooth, context-aware adaptation.

Summary updated

BibTeX
@inproceedings{dalen2009transforming,
  author = {R. C. van Dalen and F. Flego and M. J. F. Gales},
  title = {{Transforming features to compensate speech recogniser models for noise}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure