TL;DR
This paper explores using spectral features to better estimate the intelligibility of speech processed by nonlinear enhancement algorithms. While these features show promise in identifying important speech segments, they didn't outperform existing methods for objective intelligibility assessment, suggesting that current level-based weighting remains superior.
What Problem Does This Paper Address?
Estimating how intelligible speech remains after it's been processed by nonlinear enhancement algorithms (common in hearing aids and speech processing) is a significant challenge. Traditional objective intelligibility measures, like the Speech Intelligibility Index (SII) and Speech Transmission Index (STI), struggle with nonlinear processing because it violates the principle of superposition. Nonlinear processors often focus on low-level and transient speech components, which carry crucial acoustic cues and context, but are difficult to quantify in terms of their contribution to intelligibility. Subjective listening tests are laborious, making objective measures essential for algorithm development. This paper aims to improve objective intelligibility assessment by using spectral features to identify and weight these critical, information-rich speech segments.
Methods
The core of the paper involves extracting and adapting spectral features to label segments of speech with high information content. They propose using four spectral measures: Renyi Entropy (RE), Shannon Entropy (SE), Spectral Band Energy (SBE), and Madhu Flatness Measure (MF). These measures are calculated from auditory sub-bands of the speech signal. To highlight transient speech components, the first-order derivative of these spectral features is computed.
These differentiated spectral features are then used to create short-time feature vectors. These vectors are adapted to the time-course of the Coherence-based Speech Intelligibility Index (CSII) method, a more advanced objective measure that accounts for nonlinear distortions. The adaptation involves compressing or expanding the feature vectors using an exponent 'p' and applying a recursive smoothing filter.
Finally, a genetic optimization algorithm is employed to tune the parameters of these spectral feature measures (specifically, parameters 'a', 'e', and 'p') and the CSII's logistic function parameters to match psychoacoustic data from a listening test. The performance of the optimized measures is then compared against existing objective intelligibility metrics.
Datasets
The paper uses clean speech waveforms for calculating spectral features. For evaluation, a listening test was conducted with eight participants with normal hearing. This test involved semantically unpredictable sentences (SUS) in German, presented under five conditions of additive speech-shaped noise and five conditions of envelope thresholding (a strong nonlinear distortion). The objective intelligibility measures are evaluated against the subjective performance (percent correct scores) from this listening test.
Key Insights and Ideas
- Spectral Features as Intelligibility Indicators: The paper investigates whether spectral features, like entropy and flatness, can effectively identify segments of speech that are crucial for intelligibility, particularly the transient and low-level components.
- Limitations of Spectral Features for Weighting: Despite showing some correlation with speech transitions, the optimized spectral feature weighting did not lead to an improvement in objective intelligibility prediction compared to existing methods.
- The Robustness of Level-Based Weighting: The study reinforces the effectiveness of level-based weighting (e.g., CSII with RMS level weighting) for objective intelligibility assessment, even in the presence of nonlinear distortions.
- The Importance of Transient Segments: The research acknowledges that transient speech segments, though low in energy, are highly intelligible, underscoring the need for accurate methods to identify and weight them.
Key Results
The optimization process yielded specific parameter values for the spectral features (SBE, SE, MF, RE) and the CSII method. Table 1 summarizes these optimized parameters.
When comparing the performance of the proposed spectral feature-weighted CSII measures against existing methods like STOI, CSII, and level-weighted CSII, the results were not conclusive for improvement. The paper states: "As can be seen, none of the spectral feature measures yields a frame-weighting that improves the correlation with high information content as compared to the existing measures."
- Only Renyi Entropy (RE) showed a slight improvement over the unweighted CSII, but at the cost of increased standard deviation.
- The optimized level-weighted CSII measures (specifically, the one-level optimized CSII) performed very well, achieving high correlation coefficients (r²) and rank statistics (Kendall's τ). For instance, the one-level CSII achieved an r² of 0.89 and a τ of 0.87.
Limitations and Future Work
Limitations:
- The primary limitation is that the proposed spectral feature weighting methods did not outperform existing level-based weighting methods for objective intelligibility assessment.
- The study focused on specific spectral features and a particular adaptation of the CSII method. Other features or adaptations might yield different results.
Future Work:
- The authors suggest exploring whether applying spectral entropy measures to the source signal or filter signal alone (from the source-filter model) could yield better results for identifying speech transitions.
- Further investigation into how acoustic speech information rate and perceptual performance are coupled, and how information theory concepts can be integrated into speech processing, is recommended.
- The authors note that the performance of the one RMS level CSII needs to be confirmed in broader listening tests, although the STOI measure has already undergone extensive validation.
Summary updated