Speech Recognition by Denoising and Dereverberation Based on Spectral Subtraction in a Real Noisy Reverberant Environment

Kyohei Odani, Longbiao Wang, Atsuhiko KaiInterspeech 2012 · DOI

Introduction

Automatic Speech Recognition (ASR) performance often degrades significantly in "distant-talking" scenarios, such as interacting with smart speakers. Two primary factors contribute to this: background noise and reverberation (the "smearing" of sound as it reflects off surfaces). While many algorithms perform well in simulated environments, they frequently fail in real-world settings where noise is unpredictable and room acoustics are complex. This paper introduces a robust speech enhancement framework that combines multi-channel blind impulse response estimation with Generalized Spectral Subtraction (GSS) to address these challenges in real noisy and reverberant environments.

Proposed Framework

The authors propose a two-stage "Denoising and Dereverberation" (DNR) pipeline designed to clean audio signals blindly, without prior knowledge of room dimensions or speaker locations.

The proposed Denoising and Dereverberation (DNR) pipeline architecture.

The pipeline consists of several key components:

  1. Blind Impulse Response Estimation: Utilizing a Frequency-Domain Multi-Channel Least Mean Squares (MCLMS) algorithm, the system estimates the room's acoustic fingerprint (impulse response) directly from the multi-channel input.
  2. Generalized Spectral Subtraction (GSS): Unlike standard Power Spectral Subtraction, GSS employs a tunable exponent parameter ($n$), providing greater flexibility and robustness against the estimation errors common in real-world environments.
  3. Hybrid Dereverberation Strategy: Late reverberation is treated as additive noise and subtracted via GSS. Early reverberation is handled later in the pipeline through Cepstral Mean Normalization (CMN) during the feature extraction phase.
  4. Beamforming: Following denoising, signals from multiple microphones are integrated using delay-and-sum beamforming to further isolate and enhance the target speech.

Experimental Evaluation

The method was evaluated using both simulated and real-world data. Simulated tests utilized impulse responses from the RWCP and CENSREC-4 databases. For real-world validation, the researchers recorded a dataset in a 7.1m x 3.3m room featuring five male speakers and a 9-channel microphone array.

Experimental setup illustrating the table layout, speaker positions (A-E), and the 9-channel microphone array configuration.

The real-world environment included consistent background noise from an electric fan, resulting in an average Signal-to-Noise Ratio (SNR) of 18 dB. The system was tested on a Large Vocabulary Continuous Speech Recognition (LVCSR) task using the Julius decoder.

Key Results and Insights

  • Accuracy Improvements: In the real-world test environment, the GSS-based method achieved an average word accuracy of 76.2%. This represents a 39.1% relative word error reduction compared to standard CMN (60.9% accuracy) and an 11.5% reduction compared to previous Power SS methods (73.1% accuracy).
  • Real-World Robustness: A key insight was that while GSS and Power SS performed similarly in simulations, GSS showed a significant advantage in real environments. This suggests that the flexibility of GSS is essential for handling the unpredictable acoustics of actual spaces.
  • Treatment of Late Reverb: Treating late-stage echoes as additive noise proved highly effective, simplifying the complex dereverberation process into a subtraction problem in the power domain.

Conclusion

The proposed DNR framework significantly enhances ASR performance in challenging acoustic environments. By combining blind estimation with flexible spectral subtraction, the system achieves substantial error reductions that are consistent across various speaker positions. Future work will focus on addressing non-stationary "transient" noises and the "Cocktail Party Problem" involving multiple simultaneous speakers.

Summary updated

BibTeX
@inproceedings{odani2012speech,
  author = {Kyohei Odani and Longbiao Wang and Atsuhiko Kai},
  title = {{Speech Recognition by Denoising and Dereverberation Based on Spectral Subtraction in a Real Noisy Reverberant Environment}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-384},
}
Figure