Maximising objective speech intelligibility by local F0 modulation

Julián Villegas, Martin CookeInterspeech 2012

TL;DR

This paper explores how modifying the fundamental frequency (F0) of voiced speech segments can improve objective speech intelligibility. By adapting F0 locally to maximize "glimpse proportion" (a measure of signal audibility in noisy environments), the researchers found significant intelligibility gains, though at the cost of perceived speech quality.

What Problem Does This Paper Address?

Improving speech intelligibility in noisy conditions is a persistent challenge. While increasing signal loudness is an option, it can lead to listener discomfort or long-term hearing damage. Researchers have explored other acoustic modifications, like those found in "Lombard speech" (speech produced in noise), which naturally exhibits changes in F0, sound level, and spectral tilt, leading to better intelligibility. However, the specific role of F0 modifications, especially local ones, on intelligibility remains unclear. Some studies suggest negative effects, while others point to benefits in specific scenarios. This paper aims to systematically investigate how locally adjusting F0, inspired by techniques for enhancing musical consonance, impacts objective speech intelligibility.

Methods

The core of this research lies in a novel F0 modification strategy. The authors first identify voiced regions within speech utterances. For each voiced region, they explore a range of F0 adjustments (scaling by factors between 0.71 and 1.41) and select the adjustment that maximizes the "glimpse proportion" (GP). GP is an objective measure that quantifies the spectro-temporal regions where a signal is not masked by noise, serving as a predictor of intelligibility. This maximization process is inspired by methods used to reduce "roughness" in musical sounds, a psychoacoustic percept related to rapid amplitude fluctuations. The modified voiced segments are then recombined with the original unvoiced segments to create the enhanced speech. Objective intelligibility was assessed using GP, and speech quality was evaluated using PESQ.

Datasets

The study utilized 120 phonetically-balanced sentences from the Harvard lists, spoken by a male British native speaker. These sentences were mixed with two types of maskers: speech-shaped noise (SSN) and a competing female talker. Three signal-to-noise ratios (SNRs) were used for each masker, chosen to yield approximate word correct rates of 75%, 50%, and 25%.

Key Insights and Ideas

  • F0 Modulation for Intelligibility: Local F0 adjustments, guided by maximizing glimpse proportion, can indeed boost objective speech intelligibility.
  • Roughness and Glimpse Proportion Link: The paper suggests a connection between musical consonance (related to low roughness) and speech intelligibility (related to high glimpse proportion), hinting at shared underlying auditory processing principles.
  • Lower F0 is Better (for Intelligibility): Surprisingly, the optimization process consistently led to lower median F0 values in the modified speech, which resulted in higher objective intelligibility scores. This contradicts the common observation that higher F0 is associated with Lombard speech.
  • Harmonic Masking Release: The authors hypothesize that lowering F0 might increase the number of harmonics within a given spectral region, potentially unmasking some components and improving intelligibility, especially when competing with other speech or noise.
  • Quality vs. Intelligibility Trade-off: While intelligibility improved objectively, the perceived speech quality (measured by PESQ) significantly degraded.

Key Results

  • F0 modifications increased mean glimpse proportion by up to 15% depending on the masker and SNR.
  • Lower mean F0 changes generally resulted in greater glimpse proportions.
  • A three-way ANOVA revealed a significant main effect of treatment (modified vs. original) on intelligibility, with no significant interaction with masker or SNR.
  • Objective intelligibility improved across all conditions, while objective speech quality (PESQ) was reduced by an average of approximately 45%.
  • Lower pitches yielded higher average GP scores.

Limitations and Future Work

The authors acknowledge that the PESQ score degradation might be partly due to artefacts in the resynthesis process. They also point out that the glimpse proportion model, while effective, doesn't account for other masking effects (informational, cognitive loading) or the role of F0 in source separation. Future work should involve subjective intelligibility evaluations and testing with a larger variety of utterances and maskers to validate these findings. The potential for implementing these F0 modifications in speech output technology is also highlighted.

Summary updated

BibTeX
@inproceedings{villegas2012maximising,
  author = {Julián Villegas and Martin Cooke},
  title = {{Maximising objective speech intelligibility by local F0 modulation}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure