Methods for robust speech recognition in reverberant environments: a comparison

Rico Petrick, Thomas Fehér, Masashi Unoki, Rüdiger HoffmannInterspeech 2010

TL;DR

This paper compares several front-end processing methods for improving automatic speech recognition (ASR) in reverberant environments. The authors introduce and evaluate two new methods, IMTF-based dereverberation and Delay-and-Sum Beamforming (DSB), alongside previously studied techniques, finding that while IMTF shows promise in far-field conditions, DSB offers limited gains and has drawbacks for ASR in reverberation.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems often struggle in real-world environments due to reverberation, which is the persistence of sound after the original sound has stopped. This phenomenon, caused by sound waves reflecting off surfaces in a room, distorts the speech signal and significantly degrades recognition rates. The challenge is to develop robust ASR systems that can maintain acceptable performance even in the presence of varying degrees of reverberation, characterized by factors like reverberation time (T60) and speaker-to-microphone distance (SMD). This research aims to compare and evaluate different front-end signal processing techniques designed to mitigate these reverberation effects, focusing on methods suitable for practical command and control applications that require low adaptation times and computational efficiency.

Methods

The paper evaluates several front-end processing methods for ASR in reverberant environments. These include:

  • Conventional Feature Analysis (CFA): A baseline method using Mel filterbank (MFB) features.
  • Harmonicity based Feature Analysis (HFA): This method emphasizes reliable harmonic spectral components in voiced speech segments while suppressing non-harmonic components and low frequencies, aiming to improve robustness against reverberation.
  • Temporal Power Envelope Feature Analysis (TPEFA): This technique extracts temporal power envelopes (TPEs) from subbands of the speech signal, which are thought to contain key intelligibility information. It also applies low-pass filtering to these TPEs.
  • HFA + TPEFA Combination: A series connection of HFA and TPEFA, where HFA synthesizes the enhanced spectra, which are then processed by TPEFA.
  • Inverse Modulation Transfer Function (IMTF) based Dereverberation: A new method introduced for evaluation. It aims to compensate for reverberation by multiplying the reverberant TPEs with an estimated inverse modulation transfer function (MTF). The paper hypothesizes and tests that this method works well in far-field conditions but poorly in near-field conditions.
  • Delay-and-Sum Beamformer (DSB): Another new method for evaluation. This technique uses multiple microphones to combine signals, aiming to attenuate unwanted noise and reverberation. The paper investigates its effectiveness, particularly considering its frequency-dependent directivity.

Datasets

The experiments were conducted using a subset of the APOLLO corpus, specifically 1020 command phrases of 17 classes, each approximately 2 seconds long. The reverberant conditions were simulated by convolving the clean speech data with measured room impulse responses (RIRs) corresponding to various reverberation times (T60) and speaker-to-microphone distances (SMD). For the DSB experiments, new RIRs were measured in four different rooms (sound studio, living room, office room, computer laboratory) with varying SMDs.

Key Insights and Ideas

  • IMTF's Field Dependence: The IMTF method shows a strong dependence on the acoustic field. It performs well in far-field conditions but can degrade recognition rates in near-field or non-reverberant situations, suggesting a need for adaptive switching based on reverberation levels.
  • DSB's Limited Gains in Reverberation: Despite being a multi-microphone technique, DSB offers only marginal improvements in ASR performance in reverberant environments. This is attributed to its limited ability to attenuate low-frequency reverberation, which is identified as the most harmful type for ASR.
  • Frequency-Dependent Reverberation Harmfulness: The paper reiterates that high-frequency reverberation is less harmful to ASR than low-frequency reverberation. DSB's directivity pattern, which is stronger at higher frequencies, makes it less effective against the more problematic low-frequency reverberation.
  • HFA's Trade-offs: HFA can improve robustness but may slightly decrease recognition rates in clean speech conditions by removing potentially useful information.
  • Combined Approaches Show Promise: The combination of DSB and HFA (DSB output feeding into HFA) yielded the best results in the new experimental setup, suggesting that combining different front-end strategies can be beneficial.

Key Results

  • IMTF: In far-field conditions, IMTF showed improvements. However, in near-field and clean speech conditions, IMTF significantly degraded recognition rates, performing worse than TPEFA alone.
  • DSB: DSB provided a modest improvement of about 10% in the far field. However, at very close SMDs, it slightly decreased recognition rates due to geometrical near-field effects.
  • HFA: HFA performed better than DSB in reverberant environments but slightly decreased RR for clean speech.
  • DSB+HFA: The combination of DSB and HFA achieved the best results in the new experimental setup, outperforming individual methods.
  • Figure 1: Illustrates IMTF's poor performance in near-field and clean conditions, with a crossover point where it starts to outperform TPEFA in the far field.
  • Figure 2: Shows that ASR performance generally decreases with increasing T60 and SMD. DSB offers some improvement, HFA performs better, and DSB+HFA yields the best results.

Limitations and Future Work

  • Limitations:
    • The IMTF method's effectiveness is limited to far-field conditions, requiring detection of reverberation levels to avoid performance degradation.
  • DSB's performance is dependent on speaker position, which is a limitation for practical applications with moving speakers.
  • DSB requires multiple microphones, adding complexity and cost compared to single-channel methods.
  • Future Work:
    • Generate experimental results for TPEFA, HFA+TPEFA, and IMTF using the newly recorded environments.
  • Test all methods with reverberant training, building upon previous work.
  • Further investigate the potential of combining HFA+DSB and HFA+TPEFA.
  • Develop methods to detect reverberation levels to intelligently switch IMTF dereverberation on or off.

Summary updated

BibTeX
@inproceedings{petrick2010methods,
  author = {Rico Petrick and Thomas Fehér and Masashi Unoki and Rüdiger Hoffmann},
  title = {{Methods for robust speech recognition in reverberant environments: a comparison}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure