Acoustic cues for the classification of regular and irregular phonation

Kushan Surana, Janet SlifkaInterspeech 2006

TL;DR

This paper introduces a novel set of four acoustic features to automatically distinguish between regular and irregular vocal fold vibration patterns in speech. These features, derived from fundamental frequency, amplitude, and energy differences, achieve over 90% accuracy in classifying these phonation types, paving the way for improved speech recognition and synthesis systems.

What Problem Does This Paper Address?

Phonation, the process of producing sound through vocal fold vibration, can manifest in various ways. While "regular" or "modal" phonation is the most common, "irregular" phonation (also termed nonmodal or aperiodic) plays a crucial role in conveying linguistic information, marking utterance boundaries, and can even signal voice disorders. Accurately detecting and classifying these irregular phonation events is essential for enhancing speech technologies like recognition and synthesis, and for clinical applications in speech pathology. Existing methods often rely on limited speaker sets or specific phonation types, leaving a gap for robust, speaker-independent classification across a broader range of irregular phonation.

Methods

The authors propose a set of four acoustic cues designed to capture the characteristics of regular versus irregular phonation:

  • Fundamental Frequency (F0): Estimated conservatively, with a zero output if confidence is low.
  • Normalized Root Mean Square (NRMS) Amplitude: The RMS amplitude of a speech segment, normalized by the overall speech signal's RMS amplitude to account for speaker volume variations.
  • Smoothed-Energy-Difference (SED) Amplitude: This cue captures the jaggedness of the speech signal's energy envelope. It's computed by taking the difference between energy envelopes smoothed with two different window sizes (6 ms and 16 ms). Regular phonation is expected to have a smoother energy envelope, resulting in fewer peaks in the SED, while irregular phonation should exhibit more pronounced peaks.
  • Shift-Difference Amplitude (SD): Based on prior work, this measure quantifies aperiodicity. It compares shifted segments of the speech waveform to a reference segment, normalized by the reference segment's power. Lower values are expected for periodic (regular) tokens, and higher values for aperiodic (irregular) ones.

These cues are then fed into a Support Vector Machine (SVM) classifier, using a Gaussian kernel, to distinguish between regular and irregular phonation tokens.

Datasets

The study utilizes a subset of the TIMIT corpus. Specifically, speech from speakers in the "Northern" (dr1) and "New England" (dr2) dialect regions was used.

  • Training Set: 114 speakers.
  • Testing Set: 37 speakers.

The irregular tokens were hand-labeled by analyzing waveforms in both temporal and frequency domains. Regular tokens were defined as vowels that did not contain instances of irregular phonation. The dataset includes tokens from both male and female speakers and represents various utterance contexts (phrase-final, utterance-final, phrase-initial).

Key Insights and Ideas

  • Multi-faceted Cues are Crucial: No single acoustic cue perfectly separates regular and irregular phonation. Combining features from temporal (amplitude, energy difference) and frequency (F0) domains provides a more robust classification.
  • Energy Envelope Jaggedness as a Key Indicator: The Smoothed-Energy-Difference (SED) cue effectively captures the difference in the smoothness of the speech energy envelope between regular and irregular phonation, providing a strong discriminative signal.
  • Aperiodicity Measure is Effective: The Shift-Difference Amplitude (SD) cue, adapted from previous work, proves valuable in quantifying the degree of aperiodicity, a hallmark of irregular phonation.
  • Speaker Independence Achieved: The proposed method demonstrates good performance across a large number of speakers, addressing a key limitation of prior research.
  • Contextual Variation Handled: The inclusion of tokens from various utterance positions suggests the method is not overly sensitive to specific phonetic or prosodic contexts.

Key Results

The study reports impressive classification rates:

  • Irregular Phonation: 91.25% recognition rate.
  • Regular Phonation: 95.02% recognition rate.

These results were achieved on a test set comprising 320 irregular tokens and 4320 regular tokens. The authors note that the unequal size of the test set is representative of the natural occurrence of regular and irregular phonation in speech.

Limitations and Future Work

  • Misclassification Scenarios: The paper acknowledges that misclassifications occur. For irregular tokens, high F0 values can arise from oscillations between widely spaced glottal pulses. For regular tokens, low F0 can lead to insufficient peaks for reliable estimation. Similarly, NRMS can be high for irregular tokens with strong inter-pulse oscillations and low for regular tokens with decreasing amplitude.
  • Specific Token Types: The SED cue can be misleading for irregular tokens with only one or two glottal pulses, and for regular tokens from male speakers with low F0 where the shorter smoothing window might miss a full glottal pulse. The SD cue can also be high for regular tokens with changing signal amplitude.
  • Future Directions: The authors suggest improving the cues by reducing the impact of between-pulse oscillations and better detecting envelope changes. Further work could explore strategies to account for these specific misclassification scenarios.

Summary updated

BibTeX
@inproceedings{surana2006acoustic,
  author = {Kushan Surana and Janet Slifka},
  title = {{Acoustic cues for the classification of regular and irregular phonation}},
  booktitle = {Interspeech},
  year = {2006},
}
Figure