Robust speech recognition in reverberant environments using subband-based steady-state monaural and binaural suppression

Hyung‐Min Park, Matthew Maciejewski, Chanwoo Kim, Richard M. Stern2014 · DOI

TL;DR

This paper explores how to improve speech recognition in noisy, reverberant environments by combining a monaural "precedence effect" inspired processing with binaural cues. They found that combining these techniques significantly reduces word error rates, and surprisingly, the exact stage of binaural interaction within the processing pipeline has minimal impact on performance.

What Problem Does This Paper Address?

Recognizing speech in environments with echoes and reverberation is a major challenge for automatic speech recognition (ASR) systems. Human hearing, however, is remarkably adept at handling such conditions, partly due to the "precedence effect." This effect describes our auditory system's ability to prioritize the first arriving sound (the direct path from the source) and suppress later echoes, which helps maintain a stable perception of sound location and can improve intelligibility. While the precedence effect has traditionally been thought to be a binaural phenomenon, this paper investigates whether similar suppression mechanisms can occur monaurally and how combining these monaural enhancements with binaural processing can boost ASR performance in reverberant settings.

Methods

The core of the proposed method builds upon the "Steady-State Suppression" (SSF) technique, which aims to enhance speech onsets and suppress steady-state components in a monaural fashion. The paper extends this by exploring different ways to integrate binaural information with SSF processing.

Here's a breakdown:

  • Monaural SSF Processing: This involves:
  • Pre-emphasis and Short-Time Fourier Transform (STFT) of the input speech.
  • Calculating power in gammatone filterbank channels.
  • Low-pass filtering the power to estimate a "slowly-varying" component.
  • Subtracting this slowly-varying component from the original power to create a "high-pass filtered" signal that emphasizes onsets.
  • Modifying the magnitude spectrum of the original signal based on this processed power.
  • Re-synthesizing the enhanced speech.
  • Binaural Interaction: The paper investigates combining the SSF-processed signals from two microphones at three different stages:
  • Type I: After initial spectral analysis (combining raw STFT magnitudes).
  • Type II: After gammatone frequency integration (combining power estimates).
  • Type III: After SSF processing but before spectral reshaping (combining processed power estimates).
  • They also consider a simple delay-and-sum waveform combination of the final re-synthesized signals.

The key novelty lies in exploring these different integration points for binaural cues with the SSF-based onset enhancement.

Datasets

  • Training: The CMU SPHINX-III speech recognition system was trained on the DARPA Resource Management (RM) database using 1,600 utterances recorded in a quiet environment.
  • Testing: The test set consisted of 600 utterances from the same RM database.
  • Reverberation Simulation: Impulse responses were generated using the image method (Room Impulse Response software) to simulate acoustics in a rectangular room. The geometry of the virtual room is described and illustrated.

Key Insights and Ideas

  • Monaural Precedence Effect is Beneficial: The SSF processing, which mimics aspects of the precedence effect monaurally, significantly improves speech recognition in reverberant conditions.
  • Binaural + SSF Synergy: Combining SSF processing with binaural interaction yields substantial gains in recognition accuracy, far exceeding the benefits of either method alone.
  • Locus of Binaural Interaction is Flexible: Surprisingly, the stage at which binaural interaction is applied (early in spectral analysis, after power integration, or after SSF processing) has minimal impact on the overall performance improvement. This suggests a degree of robustness in the combined approach.
  • Microphone Spacing Matters: Performance improvements are observed to increase with microphone separation, with significant gains seen around 17 cm, and performance remaining stable up to 1 meter. This is interesting given that human binaural processing typically deals with path differences within about 22 cm.

Key Results

  • Combining SSF processing with binaural interaction led to relative Word Error Rate (WER) reductions of 30% and 59% compared to SSF alone and binaural processing alone, respectively.
  • SSF processing alone provided a relative WER increase of about 13% for clean speech (a small degradation, as noted in prior work).
  • The paper shows that binaural processing and SSF processing are both beneficial in reverberant environments, but their combination is clearly advantageous.

Limitations and Future Work

  • Limitations:
    • The experiments assumed the target source was along the perpendicular bisector of the line between the two microphones. While the authors mention that this can be extended to off-midline sources by compensating for relative time delays (estimated via speaker localization), this was not explicitly implemented or tested in this paper.
  • The effectiveness of binaural processing with only two microphones is noted as being limited, implying that more complex microphone arrays might offer further benefits.
  • Future Work:
    • The authors suggest that the flexibility in the stage of binaural interaction could be exploited to develop more efficient systems.
  • Extending the approach to handle sources from arbitrary directions by incorporating speaker localization for time delay estimation is a natural next step.

Summary updated

BibTeX
@misc{park2014robust,
  author = {Hyung‐Min Park and Matthew Maciejewski and Chanwoo Kim and Richard M. Stern},
  title = {{Robust speech recognition in reverberant environments using subband-based steady-state monaural and binaural suppression}},
  year = {2014},
  doi = {10.21437/interspeech.2014-153},
}
Figure