Recurrent Neural Networks for Noise Reduction in Robust ASR

Andrew L. Maas, Quoc V. Le, Tyler M. O'Neil, Oriol Vinyals, Patrick Nguyen, Andrew Y. NgInterspeech 2012 · DOI

Introduction

Automatic Speech Recognition (ASR) systems frequently struggle in real-world environments characterized by non-stationary noise, such as crowded cafes or moving vehicles. Traditional solutions typically rely on hand-engineered signal processing filters or robust acoustic modeling. However, these methods often fail to capture the complex, non-linear distortions of real-world audio.

This paper introduces the Deep Recurrent Denoising Autoencoder (DRDAE), a purely data-driven architecture designed to reconstruct clean speech features from noisy signals. By leveraging both depth and temporal recurrence, the DRDAE significantly outperforms traditional denoising methods on standard benchmarks.

The DRDAE Architecture

The proposed solution evolves through three stages of neural architecture to capture both hierarchical features and temporal context:

  1. Denoising Autoencoder (DAE): A feed-forward network that maps noisy audio frames to clean reconstructions.
  2. Recurrent Denoising Autoencoder (RDAE): Adds recurrent connections to utilize temporal context, acknowledging that speech is a sequential signal.
  3. Deep Recurrent Denoising Autoencoder (DRDAE): The final architecture stacks multiple hidden layers to learn highly non-linear mappings, with recurrence integrated into the middle layer to provide "memory" of speech patterns.
The DRDAE architecture showing multiple hidden layers and recurrent connections at the second hidden layer.

The model is trained on stereo data—aligned pairs of clean and artificially noisified utterances—using a Mean Squared Error (MSE) loss function to minimize the difference between the predicted and true clean Mel-Frequency Cepstral Coefficients (MFCCs).

Experimental Setup

The researchers evaluated the model using the Aurora2 corpus, a standard benchmark for robust ASR. The training set included 8,001 utterances corrupted by four noise types (subway, babble, car, exhibition) at various Signal-to-Noise Ratios (SNRs). Evaluation was conducted on two sets:

  • Test Set A: Contains the same noise types as the training set to evaluate performance on "seen" noise.
  • Test Set B: Features four entirely different noise types to test the model's ability to generalize to "unseen" distortions.

Performance and Key Insights

The DRDAE demonstrated superior resilience, particularly in high-noise conditions. A key finding was that the combination of depth and recurrence is vital; depth allows for complex denoising functions, while recurrence tracks speech patterns over time.

Comparison of Word Error Rates (WER) across different SNR levels for standard MFCCs, AFE, Tandem, and the proposed DRDAE.

As shown in the results table, the DRDAE maintains significantly lower Word Error Rates (WER) as noise increases. At a 5dB SNR, the DRDAE achieved a WER of 10.68%, compared to 14.16% for the industry-standard ETSI Advanced Front End (AFE) and a staggering 64.25% for baseline MFCCs. Overall, on Test Set A, the DRDAE reached an average WER of 10.85%, surpassing both the ETSI AFE (12.26%) and the SPLICE algorithm (11.67%).

Conclusion and Future Directions

While the DRDAE sets a new bar for feature denoising, challenges remain. The model's performance on unseen noise (Test Set B) suggests a dependency on the variety of noise encountered during training. Future work will likely focus on training with larger, more diverse datasets to improve "out-of-the-box" generalization. Additionally, while deep recurrent models are computationally intensive, their ability to act as a noise-agnostic pre-processor makes them a powerful tool for modern robust ASR systems.

Summary updated

BibTeX
@inproceedings{maas2012recurrent,
  author = {Andrew L. Maas and Quoc V. Le and Tyler M. O'Neil and Oriol Vinyals and Patrick Nguyen and Andrew Y. Ng},
  title = {{Recurrent Neural Networks for Noise Reduction in Robust ASR}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-6},
}
Figure