Detecting non-modal phonation in telephone speech

Tae-Jin Yoon, Jennifer Cole, Mark Hasegawa‐Johnson2008 · DOI

TL;DR

This paper tackles the challenge of detecting "creaky voice" (a non-modal phonation type) in low-quality telephone speech. They achieve this by developing a more robust method for analyzing fundamental frequency (F0) and spectral characteristics, outperforming traditional methods on noisy telephone recordings and achieving 75% accuracy in classifying creaky vs. modal voice.

What Problem Does This Paper Address?

Speech carries a wealth of information beyond just the words spoken, including emotional state and emphasis, often conveyed through variations in voice quality. "Non-modal phonation," such as creaky voice, is one such variation. Detecting these nuances is crucial for advanced speech processing tasks like emotion recognition or prosody analysis. However, analyzing voice quality, particularly creaky voice, relies heavily on accurately measuring the fundamental frequency (F0) and spectral properties of the voice. This becomes extremely difficult with telephone-quality speech, which is characterized by limited bandwidth and noise, often causing standard F0 analysis tools to fail. This paper aims to bridge this gap by proposing a method that can reliably extract these crucial acoustic features from challenging telephone speech data.

Methods

The researchers developed a two-pronged approach to extract relevant acoustic features:

  1. Robust F0 Extraction: Traditional F0 analysis struggles with telephone speech. Their solution involves:
  2. Intensity Normalization: Boosting the signal's intensity to overcome low-volume segments.
  3. Inverse Filtering: Applying a filter to the signal to better isolate the glottal source characteristics.
  4. Autocorrelation-based F0: Using an autocorrelation algorithm on the processed signal, which is more resilient to noise and distortion in non-modal regions where standard methods fail.
  5. Spectral Feature Extraction: Beyond F0, they analyzed spectral characteristics:
  6. Harmonic Amplitudes: Measuring the amplitude of the first two harmonics (H1 and H2).
  7. Formant Amplitudes: Measuring the amplitude of the first three formants (A1, A2, and A3).
  8. Spectral Slope: Analyzing the overall spectral tilt.
  9. These features were extracted using Fast Fourier Transform (FFT) and Long-Term Average Spectrum (LTAS) on the intensity-normalized signal.

Finally, they used Support Vector Machines (SVM) with a radial basis function kernel to classify segments as either creaky or modal voice, based on the extracted F0 and spectral features.

Datasets

The study utilized the Switchboard corpus, specifically a subset (WS97) of spontaneous telephone conversations. Approximately 200 files were identified as containing regions of non-modal, creaky phonation. From these, 160 intervals of modal phonation and 140 intervals of creaky phonation were selected for acoustic feature analysis and classification. The selection process involved manual inspection of spectrograms, waveforms, and listening to identify these phonation types.

Key Insights and Ideas

  • F0 is Tricky, but Not Impossible: The paper highlights that while F0 analysis is notoriously difficult in telephone speech, specific signal processing techniques (intensity normalization and inverse filtering) can make it more robust, especially in challenging non-modal regions.
  • Spectral Cues are Key: The research reinforces that spectral features, particularly the relative amplitudes of lower harmonics (like H1 and H2) and formants, are more reliable indicators of creaky voice than F0 alone, especially in noisy environments.
  • Creaky Voice has Distinct Spectral Signatures: The findings align with previous laboratory studies, suggesting that creaky voice is characterized by specific patterns in harmonic and formant amplitudes, even when extracted from low-fidelity recordings.
  • SVMs Can Learn These Patterns: The successful application of SVMs demonstrates that machine learning models can effectively learn the subtle acoustic differences between creaky and modal voice from the extracted features, paving the way for automated detection.

Key Results

  • Robust Feature Extraction: The proposed method successfully extracted F0 and spectral features from telephone speech where traditional methods often failed.
  • Statistical Significance: One-way ANOVA tests showed a statistically significant effect of phonation type (creaky vs. modal) on F0, H1, H1-H2, H1-A1, H1-A2, H1-A3, and spectral slope. Notably, H2 alone did not show a significant effect.
  • Classification Accuracy: The SVM classifier achieved 75% accuracy in distinguishing between creaky and modal voice. This represents a significant improvement over a baseline accuracy of 53%.

Limitations and Future Work

  • Distinguishing Similar Non-Modal Types: The authors acknowledge that creaky phonation shares spectral similarities with other non-modal types like "tense" or "pressed" phonation. Distinguishing these subtypes based solely on the current frequency-domain features remains a challenge.
  • Expanding Feature Set: Future work will involve incorporating time-domain features to better differentiate creaky phonation from tense phonation.
  • Unsupervised Learning: The researchers are exploring clustering algorithms to automatically learn distinct voice quality clusters without explicit labels, which could be valuable for large, unlabeled speech corpora.
  • Broader Applications: The ultimate goal is to use these non-modal phonation detection capabilities for recognizing prosodic events (like accents and junctures) and disfluencies in speech.

Summary updated

Abstract

Non-modal phonation conveys both linguistic and paralinguistic information, and is distinguished by acoustic source and filter features.Detecting non-modal phonation in speech requires reliable F0 analysis, a problem for telephone-band speech, where F0 analysis frequently fails.We demonstrate an approach to the detection of creaky phonation in telephone speech based on robust F0 and spectral analysis.Our F0 analysis relies on an autocorrelation algorithm applied to the intensity-boosted and inverse-filtered speech signal and succeeds in regions of nonmodal phonation where the non-filtered F0 analysis typically fails.In addition to the extracted F0 values, spectral amplitude is measured at the first two harmonics (H1, H2) and the first three formants (A1, A2, A3).Visual and spectral inspection of the detected creaky phonation confirms the findings reported from laboratory setting.Statistical analysis using oneway ANOVA and classification using Support Vector Machine (SVM) reveals promising results which lead to further improvement for automatic detection of non-modal phonation in telephone speech.
BibTeX
@misc{yoon2008detecting,
  author = {Tae-Jin Yoon and Jennifer Cole and Mark Hasegawa‐Johnson},
  title = {{Detecting non-modal phonation in telephone speech}},
  year = {2008},
  doi = {10.21437/speechprosody.2008-7},
}
Figure