A glimpse-based approach for predicting binaural intelligibility with single and multiple maskers in anechoic conditions

Yan Tang, Martin Cooke, Bruno M. Fazenda, Trevor J. CoxInterspeech 2015

TL;DR

This paper extends a monaural speech intelligibility prediction metric to the binaural domain, accounting for the "better ear" effect and binaural unmasking. The new metric, BiDWGP, accurately predicts human intelligibility in complex listening scenarios with single and multiple maskers, using only monaural signals and source locations. This offers a practical tool for audio engineers and researchers to assess and design for speech intelligibility without requiring full binaural audio rendering.

What Problem Does This Paper Address?

Predicting how well a person will understand speech in noisy environments is crucial for developing better hearing aids, audio processing algorithms, and even designing immersive audio experiences. While objective intelligibility measures (OIMs) exist for monaural (single-ear) listening, predicting intelligibility in binaural (two-ear) listening, especially with multiple competing sounds, is more challenging. Existing binaural OIMs often require complex binaural audio signals or impulse responses, which aren't always available during the design phase. This paper tackles the problem of predicting binaural speech intelligibility using only monaural speech and masker signals, along with their spatial locations, making it more practical for real-world applications.

Methods

The core of this work is the extension of a monaural "glimpse-based" intelligibility metric, the Distortion-Weighted Glimpse Proportion (DWGP), into the binaural domain, creating the Binaural DWGP (BiDWGP). The DWGP metric works by identifying "glimpses" – time-frequency regions where speech is perceptually available despite masking. It incorporates distortion weighting to account for how maskers distort the speech envelope.

The BiDWGP introduces two key binaural aspects:

  1. Binaural Unmasking: This accounts for the phenomenon where listeners can better understand speech when the masker is different from the speech (e.g., different interaural time differences). This is incorporated using a measure derived from the binaural masking level difference (BMLD).
  2. Better Ear Effect: This acknowledges that listeners naturally favor the ear receiving the clearer signal. BiDWGP simulates this by considering a "glimpse" to exist if it's present in either the left or right ear's auditory representation.

The model processes monaural speech and masker signals through a simulated auditory periphery (gammatone filters), calculates spectro-temporal excitation patterns (STEPs) for each ear, and then applies the binaural unmasking and better ear effects to define binaural glimpses. Finally, these glimpses are weighted by frequency importance and combined to produce a single intelligibility score between 0 and 1.

Datasets

The paper does not introduce new datasets. Instead, it uses existing datasets for evaluation:

  • Speech: Harvard sentences uttered by a British English male talker.
  • Maskers:
    • Stationary Noise (SSN): Speech-shaped noise.
  • Competing Speech (CS): Sentences uttered by a British English female talker from the SCRIBE corpus.
  • Head-Related Impulse Responses (HRIRs): Recorded in an anechoic chamber to simulate spatial audio.

The experiments were conducted with 14 native British English speakers.

Key Insights and Ideas

  • Practicality of Monural Inputs: The most significant insight is that accurate binaural intelligibility prediction is achievable using only monaural signals and source locations. This bypasses the need for complex binaural audio rendering, making the metric highly practical for audio design tools.
  • Effective Binaural Modeling: The BiDWGP successfully integrates the "better ear" effect and binaural unmasking into a single framework, demonstrating their importance in binaural listening.
  • Robustness to Masker Type: The model shows comparable predictive accuracy for both stationary noise and fluctuating competing speech maskers, which is a notable improvement over some monaural OIMs that struggle with fluctuating noise.
  • Overestimation in Complex Scenarios: The model tends to overestimate intelligibility when multiple competing speech maskers are present, particularly when listener scores are low. This suggests that factors beyond simple signal-to-noise ratios, such as attention switching for source localization and segregation, play a significant role in these challenging conditions.

Key Results

The BiDWGP metric demonstrated strong performance in predicting subjective intelligibility scores:

  • Single Masker Conditions: The model achieved a very high correlation of p = 0.97 with listener scores. For individual masker types (SSN and CS), the correlations were even higher (p = 0.98).
  • Multiple Masker Conditions: The overall correlation dropped slightly to p = 0.91. For individual masker types in these conditions, correlations were p = 0.96 for SSN and p = 0.94 for CS.
  • Comparison to Binaural SII: The BiDWGP significantly outperformed the binaural Speech Intelligibility Index (SII) in single masker conditions (p=0.97 vs. p=0.71).

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Overestimation with Multiple Competing Speech Maskers: The model tends to overestimate intelligibility in complex scenarios with multiple competing speech maskers, especially when listener performance is low. This is attributed to potential listener attention switching for source localization and segregation.
  • Exclusion of Reverberation: The current BiDWGP metric is limited to anechoic conditions and does not account for the effects of reverberation.
  • Future Work:
    • Investigate the "cost of attention-switching" hypothesis through experiments with visual cues.
  • Explore the impact of informational masking from competing speech.
  • Extend the BiDWGP metric to handle reverberant acoustic conditions.

Summary updated

BibTeX
@inproceedings{tang2015glimpse,
  author = {Yan Tang and Martin Cooke and Bruno M. Fazenda and Trevor J. Cox},
  title = {{A glimpse-based approach for predicting binaural intelligibility with single and multiple maskers in anechoic conditions}},
  booktitle = {Interspeech},
  year = {2015},
}
Figure