A computational model of the speech reception threshold for laterally separated speech and noise

Guy J. Brown, Kalle J. PalomäkiInterspeech 2005

TL;DR

This paper introduces a computational model that mimics human ability to separate speech from noise based on their spatial separation, even when the cues are inconsistent across frequencies. The model achieves this by combining binaural processing with a "missing data" approach to automatic speech recognition, showing promising qualitative agreement with human performance in challenging listening scenarios.

What Problem Does This Paper Address?

Understanding how humans separate sounds in complex auditory environments, particularly speech from interfering noise, is a long-standing challenge in auditory perception and signal processing. A key aspect of this is spatial separation, where sounds originating from different locations are easier to distinguish. Many computational models of auditory scene analysis (CASA) assume that humans group sounds based on consistent interaural time differences (ITDs) across all frequencies. However, recent psychophysical experiments suggest humans can still segregate sounds even when these ITDs are inconsistent or "swapped" between frequency bands. This paper addresses the gap between existing CASA models and these human listening capabilities, aiming to develop a model that can handle such complex, inconsistent spatial cues.

Methods

The authors propose a computational model that integrates three main components:

  • Peripheral Auditory Model: This stage simulates the human ear's initial processing using gammatone filters to analyze sounds across different frequency bands. It also simulates auditory nerve activity.
  • Mask Estimation: This is the core innovation. The model estimates a "mask" that tells a speech recognizer which parts of the signal are likely to be speech and which are noise. This mask is derived from two sources:
  • Binaural Equalization-Cancellation (EC): This mechanism exploits differences in ITD between the speech and noise. It attempts to cancel out the noise based on its estimated ITD within each frequency channel.
  • Fundamental Frequency (F0) based Grouping: This uses the harmonic structure of speech (pitch) to identify and select speech components. The model combines these two cues using a logical OR operation. Crucially, the binaural EC mechanism operates within each frequency channel, allowing it to handle inconsistent ITDs.

  • Missing Data ASR: The estimated mask is fed into a "missing data" automatic speech recognition (ASR) system. This ASR framework is designed to handle situations where parts of the acoustic signal are unreliable or missing, treating them differently during recognition.

Datasets

  • TiDigits Corpus: This corpus of spoken digits was used for training the acoustic models of the ASR system.
  • Edmonds' Stimuli (adapted): The experimental stimuli, consisting of speech and speech-shaped noise with manipulated ITDs across frequency bands, were based on those used in a previous psychophysical study by Edmonds [4]. The authors adapted these stimuli for their ASR system.

Key Insights and Ideas

  • Beyond Consistent ITDs: The most significant insight is that human sound segregation doesn't strictly require consistent ITDs across all frequencies. The model's ability to handle "swapped" ITDs demonstrates a more flexible approach to spatial cue utilization.
  • Within-Channel Processing is Key: The success of the "within-channel EC model" highlights the importance of processing binaural cues independently for each frequency band to account for inconsistent spatial information.
  • Synergy of Binaural and Pitch Cues: The model effectively combines binaural cancellation (based on ITD) with F0-based grouping. This suggests that humans likely use multiple cues in parallel to achieve robust speech segregation.
  • "Missing Data" ASR as a Powerful Framework: The use of a "missing data" ASR system proves effective for integrating the output of the auditory scene analysis model, allowing it to perform speech recognition even with imperfect or incomplete acoustic information.

Key Results

  • Qualitative Agreement with Humans: The within-channel EC model showed a qualitative match to human performance in the speech reception threshold (SRT) tests. Specifically, it demonstrated a similar benefit from the presence of a spatial separation between speech and noise, regardless of whether the ITDs were consistent or swapped across frequency bands.
  • Higher SRTS than Humans: The computational model generally exhibited higher SRTs (meaning it required more speech power relative to noise to achieve 50% intelligibility) compared to human listeners, indicating that human auditory processing is still more efficient.
  • Across-Channel Model Failure: In contrast to the within-channel model, an "across-channel" model (which assumes consistent ITDs across frequencies) performed poorly when ITDs were swapped, failing to replicate human performance.

Limitations and Future Work

  • Quantitative Gap: The authors acknowledge that their model's SRTs are substantially higher than human listeners', indicating room for improvement in modeling the efficiency of human auditory processing.
  • Realistic Spatialization: Future work will involve using more realistic head-related transfer functions (HRTFs) for spatializing sounds and testing the model with a wider variety of speech and noise mixtures.
  • Engineering Applications: The authors aim to explore how a more accurate model of human processing can lead to improved performance in engineering applications, such as noise reduction and speech enhancement systems.

Summary updated

BibTeX
@inproceedings{brown2005computational,
  author = {Guy J. Brown and Kalle J. Palomäki},
  title = {{A computational model of the speech reception threshold for laterally separated speech and noise}},
  booktitle = {Interspeech},
  year = {2005},
}
Figure