TL;DR
This paper introduces a novel Resonator-based Creaky Voice Detection (RCVD) method that effectively identifies creaky voice segments in speech by analyzing secondary excitation peaks in the linear prediction residual. This approach significantly outperforms existing state-of-the-art methods, offering a more robust way to capture valuable paralinguistic information often missed by traditional speech analysis.
What Problem Does This Paper Address?
Creaky voice, a distinctive non-modal voice quality, is frequently used by speakers for expressive and interactive purposes, conveying nuances beyond the literal meaning of words. This includes signaling turn-taking, hesitations, and emotional states. However, traditional speech analysis techniques, such as fundamental frequency (Fo) tracking and spectral analysis, often struggle to accurately detect creaky voice. This is due to the extremely low Fo values and long glottal closed phases characteristic of creak, which can fall outside the typical analysis parameters or lead to spurious acoustic values. Consequently, many speech technology applications discard these segments, losing valuable paralinguistic information. The paper aims to address this by developing an automatic and accurate method for creaky voice detection.
Methods
The core of the proposed Resonator-based Creaky Voice Detection (RCVD) method lies in analyzing the linear prediction (LP) residual signal. The authors observed that creaky voice production, often involving ventricular fold incursions, creates secondary excitation peaks in this residual signal.
The RCVD method works as follows:
- LP Residual Generation: The speech signal is processed using Linear Predictive Coding (LPC) inverse filtering to obtain the residual signal.
- Resonator Excitation: This residual signal is then used to excite two complex-conjugate pole resonators.
- Resonator 1: This resonator is tuned to estimate the fundamental frequency (Fo) contour, with a bandwidth set to capture variations in creaky voice. It uses a corrected autocorrelation function to determine local glottal periods.
- Resonator 2: This resonator is specifically designed to highlight secondary excitations. It has a narrower bandwidth (150 Hz) to amplify the harmonic structure resulting from these secondary peaks.
- Harmonic Measurement: The significance of secondary excitations is measured by analyzing the spectrum of Resonator 2's output. Specifically, the difference in decibels between the first two harmonics (H2-H1) is computed. This metric is then smoothed with a 100 ms moving average filter to reduce the impact of outliers.
- Thresholding: A threshold is applied to the H2-H1 contour. When H2-H1 values rise significantly above a baseline (e.g., 0 dB), the segment is classified as creaky.
- Post-Processing (Optional): An additional post-processing step is included to remove false detections in silent and unvoiced regions by analyzing signal energy and zero-crossing rates.
Datasets
The evaluation of the RCVD method was performed on a diverse set of speech databases:
- Databases containing creak:
- Finnish male speaker (MV)
- Finnish female speaker (HS)
- American English male speaker (ARCTIC-BDL)
- Databases without creak (used for evaluating false alarms):
- Scottish male speaker (ARCTIC-AWB)
- American female speaker (ARCTIC-SLT)
Fifty sentences from each database were used, totaling 250 sentences. All speech data were downsampled to a sampling frequency of 16 kHz. Human annotation of creaky regions was performed by the first two authors using auditory criteria and waveform/spectrogram inspection.
Key Insights and Ideas
- Secondary Excitations as a Creak Signature: The paper's central insight is that secondary excitation peaks in the LP residual, likely caused by ventricular fold involvement, are a reliable acoustic correlate of creaky voice.
- Resonator for Harmonic Amplification: Using a resonator tuned to the LP residual effectively amplifies the harmonic structure produced by these secondary excitations, making them easier to detect and quantify.
- H2-H1 as a Discriminative Feature: The H2-H1 harmonic difference proves to be a sensitive metric for distinguishing creaky voice from normal phonation, as creaky voice exhibits a richer harmonic content.
- Importance of Post-Processing: The inclusion of post-processing to remove false alarms in non-speech regions significantly improves the overall performance, particularly reducing false positives without substantially increasing missed detections.
Key Results
The RCVD method, especially when combined with post-processing (RCVD+PP), demonstrated superior performance compared to two existing state-of-the-art methods (Ishi's method and the APP detector) across all evaluated databases.
- Frame-level metrics: RCVD consistently achieved higher True Positive Rates (TPR) and F1 scores, with significantly lower False Positive Rates (FPR) compared to the comparison algorithms. For instance, on the MV dataset, Ishi's method showed a surprisingly high number of false positives.
- Event-level metrics: RCVD+PP produced the fewest misses and false alarms. While the APP detector sometimes detected very short segments of irregular phonation, it often missed larger portions of creaky regions, leading to a low F1 score.
- Threshold Impact: The paper found that a threshold of 0 dB for the H2-H1 metric provided consistent performance across databases, though further analysis on larger datasets is suggested for optimal threshold setting.
Limitations and Future Work
- Threshold Optimization: While a 0 dB threshold performed well, the authors suggest that further analysis on larger datasets is needed to fully explore the optimal threshold setting for the H2-H1 metric.
- APP Detector's Scope: The APP method's broader scope of detecting "irregular phonation" (including creak, vocal fry, diplophonia, etc.) might explain its lower performance specifically for creaky voice detection, as it may be less specialized.
- Future Work: The paper implicitly suggests that the RCVD+PP method provides a strong foundation for future work in paralinguistic speech analysis. Further research could focus on refining the thresholding mechanism and exploring the application of this robust creaky voice detection in various speech technology domains, such as text-to-speech synthesis and emotion recognition.
Summary updated