Introduction
For individuals using Cochlear Implants (CIs), navigating real-world acoustic environments is a significant challenge. Background noise and reverberation (echoes) can severely degrade speech intelligibility. While engineers continuously develop speech enhancement algorithms to mitigate these effects, testing them typically requires subjective evaluations with CI users. This process is time-consuming, expensive, and difficult to scale.
This study, presented at INTERSPEECH 2012, evaluates whether standard objective speech metrics—mathematical models designed to predict human perception—can reliably substitute for subjective testing in the context of CI users.
Methodology
The researchers evaluated six "intrusive" objective metrics, which compare a degraded audio signal against a clean reference to calculate a score:
- Quality Estimators: PESQ (Perceptual Evaluation of Speech Quality), oPESQ (optimized for reverberation), KLD (Kullback-Leibler Distance), and FWSSRR (Frequency-Weighted Segmental Signal-to-Reconstruction Ratio).
- Intelligibility Predictors: NCM (Normalized Covariance Metric) and CSII (Coherence Speech Intelligibility Index).
To better reflect the hearing limitations of CI users, who have a narrower dynamic range than normal-hearing individuals, the researchers also tested modified versions of these metrics. Specifically, they narrowed the internal Signal-to-Noise Ratio (SNR) window of the algorithms (e.g., from the standard 30dB range down to a 10dB range) to emulate CI processing.
Experimental Setup
The study utilized a subjective database involving 11 adult CI users listening to sentences from the IEEE corpus. The audio was processed under 12 distinct environmental conditions:
- Noise-only: Speech-shaped noise at four SNR levels (-5 to 10 dB).
- Reverberation-only: Four room echo durations (RT60 from 0.3s to 1.0s).
- Combined: Scenarios featuring both noise and reverberation, mimicking complex real-world settings like crowded restaurants.
Results and Analysis
The evaluation revealed that while most metrics are effective at predicting the impact of background noise, their performance degrades significantly in the presence of reverberation.
The Impact of Reverberation
Reverberation proved to be a major obstacle for objective metrics. In "noise-plus-reverberation" scenarios, most metrics failed to correlate with human performance. For instance, the industry-standard PESQ metric saw its correlation coefficient ($\rho$) drop to a negligible 0.08 in combined conditions, making it effectively useless for evaluating CI speech enhancement in those environments.
NCM: The Most Robust Predictor
The Normalized Covariance Metric (NCM) emerged as the most reliable tool across all tested conditions. NCM achieved high Pearson correlations ($\rho \approx 0.93$ to $0.94$ overall), likely because it focuses on the temporal envelope of the speech signal, which is the primary information transmitted by CI processors.
Tuning for CI Limitations
Modifying the metrics to account for the reduced dynamic range of CI users provided modest improvements for some metrics like FWSSRR. However, these adjustments did not fully resolve the inaccuracies observed in complex reverberant environments, suggesting that simple SNR windowing is not a complete solution for modeling impaired hearing.
Conclusion
The study highlights a critical gap in current objective speech evaluation tools for CI applications. While metrics like NCM and CSII show promise, standard quality metrics like PESQ are effectively useless in complex, real-world acoustic scenarios involving both noise and reverberation.
Future research should focus on developing metrics that explicitly model CI signal processing strategies (such as ACE or CIS) and the unique characteristics of electrical stimulation of the auditory nerve, rather than merely adapting models built for acoustic hearing.
Summary updated