TL;DR
This paper investigates how people naturally alter their speech in noisy environments (the Lombard effect) and uses these insights to develop a novel method for transforming normal speech to improve its intelligibility, even outperforming Lombard speech itself in objective intelligibility tests. The work highlights that the intelligibility boost from the Lombard effect is primarily associated with voiced speech segments.
What Problem Does This Paper Address?
When speaking in noisy environments, humans instinctively modify their speech to enhance intelligibility. This phenomenon, known as the Lombard effect, involves changes in pitch, speaking rate, and spectral characteristics. While the Lombard effect is well-documented, understanding its precise acoustic contributions to intelligibility, especially in an unsupervised manner (without pre-labeled speech segments), remains an area of interest. Furthermore, there's a growing need to develop speech technologies, like synthesizers, that are inherently more intelligible in noisy conditions. This paper aims to bridge this gap by analyzing the Lombard effect using generalized acoustic features and then leveraging these findings to create a transformation technique that improves speech intelligibility.
Methods
The researchers employed an unsupervised approach to analyze both normal and Lombard speech. They used two primary acoustic indicators:
- Voicing Parameter: Estimated by counting zero-crossings in a speech frame, this helps distinguish between voiced (e.g., vowels) and unvoiced (e.g., consonants) speech segments.
- Stationarity Parameter: This measures the rate of change in spectral envelope variations, differentiating between stable (stationary) speech and transitional segments.
For evaluation, they used two objective metrics:
- Loudness: Based on the PEAQ standard, focusing on the 500-4500Hz frequency range, which is more representative of human hearing sensitivity.
- Speech Intelligibility Index (SII): An extended version based on ANSI standards, calculated with added white and babble noise at different signal-to-noise ratios (SNRs).
A key novel contribution is the spectral envelope transformation. Inspired by voice conversion techniques, they estimated speaker-dependent "correction filters." These filters represent the log-difference between the average spectral envelopes of Lombard and normal speech. The DC component of these filters was set to zero to preserve frame energy. An "enhanced" version of these filters was also developed, specifically trained on Lombard speech segments with high intelligibility (SII > 0.6). These filters were then applied to normal speech to modify its spectral envelope and, consequently, improve its intelligibility.
Datasets
The speech data used in this study comes from the GRID corpora. This corpus includes recordings from 8 British English speakers (4 male, 4 female). For each speaker, 50 sentences were recorded in two conditions:
- Normal: Recorded in quiet conditions.
- Lombard: Recorded in speech-shaped noise at a 96dB level.
The Lombard speech was time-aligned with the normal speech using the VoCALign software. The paper does not introduce a new benchmark dataset but rather utilizes an existing, well-established corpus for speech analysis.
Key Insights and Ideas
- Lombard Effect is Voiced-Centric: The intelligibility and loudness improvements associated with the Lombard effect are primarily observed in voiced speech segments. Unvoiced segments, in contrast, become objectively less loud and potentially less intelligible in Lombard speech compared to normal speech.
- Stationarity Doesn't Differentiate Lombard Effect: While transient speech segments are generally less intelligible than stationary ones, the Lombard effect itself doesn't seem to specifically target or differentiate between stationary and transient speech in terms of loudness or intelligibility changes.
- Spectral Envelope Transformation for Intelligibility: Modifying the spectral envelope of normal speech using speaker-specific correction filters can significantly boost objective intelligibility. This technique effectively "enhances" normal speech towards a more intelligible state.
- "Enhanced" Filters Outperform: Using correction filters trained on the most intelligible Lombard speech segments ("enhanced" filters) yields even greater improvements in objective intelligibility than using filters trained on all Lombard speech.
- Frequency Dynamic Range Compression: The correction filters act as a form of speaker-dependent frequency dynamic range compression, concentrating energy within the crucial 500-4500Hz range.
Key Results
- High Correlation: Loudness and SII metrics showed a strong positive correlation (Pearson correlation coefficient of 0.88 for Normal and 0.92 for Lombard speech), suggesting they can be used interchangeably to some extent.
- Lombard Speech Characteristics: Lombard speech exhibited higher median Loudness and SII values compared to Normal speech, particularly for voiced segments.
- Unvoiced Speech Degradation: Unvoiced Lombard speech was objectively less loud and less intelligible than unvoiced Normal speech.
- Transformed Speech Superiority: The spectral envelope transformation of Normal speech using the "enhanced" correction filters resulted in objective intelligibility (SII) that was higher than both Normal and even the original Lombard speech.
- Median SII for Normal speech: 0.34
- Median SII for Lombard speech: 0.41
- Median SII for Transformed speech (with correction filters): 0.42
- Median SII for Transformed speech (with enhanced correction filters): 0.45
Limitations and Future Work
- Subjective vs. Objective Intelligibility: The paper acknowledges that while objective metrics like SII are informative, their direct correlation with subjective listener perception, especially for unvoiced sounds (like plosives), needs further investigation. Informal listening tests suggest improved intelligibility, but formal subjective evaluations are ongoing.
- Unvoiced Sound Intelligibility: The current objective metrics might not fully capture the intelligibility of unvoiced sounds, which can rely on localized "noisy" energy bursts.
- Future Work: The authors propose future research to:
- Localize specific spectro-temporal regions contributing to the Lombard effect.
- Develop more precise transformation techniques based on these localized characteristics.
- Continue unsupervised approaches for speech modification.
Summary updated