Analysis of nonmodal phonation using minimum entropy deconvolution

Nicolas Malyska, Thomas F. QuatieriInterspeech 2006

TL;DR

This paper explores using Minimum Entropy Deconvolution (MED), a signal processing technique, to better understand "nonmodal phonation" – the irregular, creaky, or fry-like sounds in speech. By decomposing these sounds into a series of sharp impulses and their corresponding vocal tract responses, the authors show MED can reveal crucial information for speech and speaker recognition, outperforming traditional linear prediction in certain scenarios.

What Problem Does This Paper Address?

Speech isn't always smooth and melodic. Sometimes, our vocal folds vibrate irregularly, leading to sounds described as "creak," "vocal fry," or "diplophonia." These "nonmodal phonation" events, while common even in normal speech (occurring in up to 44% of word-initial vowels in one study), pose a challenge for analysis. Understanding the precise timing and characteristics of the individual glottal pulses within these nonmodal regions is important because these features can encode information about the speaker, language, dialect, and even vocal fold health. Traditional methods for analyzing speech signals, like linear prediction, struggle to accurately decompose these complex, non-uniform pulse patterns, especially when the underlying vocal tract system is "mixed-phase" (meaning it has both forward and backward acting components). This paper aims to address this gap by investigating the effectiveness of Minimum Entropy Deconvolution (MED) for analyzing nonmodal phonation and improving the estimation of the underlying speech production model.

Methods

The core of this paper is the application and evaluation of Minimum Entropy Deconvolution (MED) for analyzing nonmodal speech. MED is a signal processing technique that, unlike linear prediction (which maximizes residual entropy), aims to minimize disorder in the output. It achieves this by finding a filter that produces the "most pulse-like" residual signal. This is quantified by maximizing the "varimax norm," a measure sensitive to sharp impulses.

The authors applied MED to both synthetic and natural nonmodal speech signals. For analysis, they used overlapping 20-ms Hamming windows. They also introduced a novel hybrid approach that combines MED with conventional linear prediction. This hybrid method leverages linear prediction's ability to handle mixed-phase systems and then uses MED's varimax norm criterion to select the best configuration of pole/zero locations, aiming for a more pulse-like residual and thus a more accurate estimation of the composite impulse response (which models the vocal tract and radiation characteristics).

Datasets

The paper uses two types of data:

  • Synthetic nonmodal speech: Generated to have known underlying vocal tract and source characteristics, allowing for quantitative evaluation of the methods.
  • Natural nonmodal speech: Collected from normal speakers, specifically from utterances containing nonmodal phonation (e.g., the end of the word "umbrella").

No new benchmarks were introduced; the focus was on evaluating existing techniques on these datasets.

Key Insights and Ideas

  • MED's Pulse-Like Advantage: MED consistently produces a more "pulse-like" residual signal compared to linear prediction, both objectively (higher varimax norm) and subjectively (visually less "ringing" or activity around the main impulses).
  • Mixed-Phase Systems are Key: The ability of MED and the proposed hybrid method to handle mixed-phase systems is crucial for accurately modeling the composite impulse response of the vocal tract, which is often mixed-phase. Linear prediction, in its standard form, is limited to minimum-phase systems.
  • Hybrid Method Balances Strengths: The hybrid linear prediction/MED approach offers a compelling balance. It inherits the robustness of linear prediction in handling mixed-phase systems while incorporating MED's pulse-likeness criterion to refine the impulse response estimation.
  • Robustness Matters: While MED excels at producing pulse-like residuals, it can sometimes produce unrealistic, overly "ringing" impulse responses, especially when the excitation is near the analysis window's edge or in the presence of noise. The hybrid method appears more robust to these issues.

Key Results

  • Varimax Norm: For synthetic nonmodal speech, MED achieved an average varimax norm of 0.66, compared to 0.34 for linear prediction and 0.04 for the input signal. Similar trends were observed for natural speech.
  • Mean-Squared Error (MSE) for Synthetic Speech: MED yielded a lower MSE (0.18 on average) when fitting the known composite impulse response compared to linear prediction (0.44). The hybrid method also showed improved MSE over standard linear prediction.
  • Qualitative Improvements: Figures demonstrate that MED and the hybrid method produce visually sharper and more distinct impulse-like residuals than linear prediction. The hybrid method also showed a better ability to represent negative peaks in the impulse response, which MED sometimes missed or distorted.
  • Noise Robustness: In experiments with added aspiration noise, the hybrid method showed a smaller increase in mean-squared error (around 0.35) compared to MED (over 0.6) as noise levels increased, indicating better robustness.

Limitations and Future Work

  • MED's "Ringing" Issue: The authors acknowledge that MED can occasionally produce unrealistic composite impulse responses that "ring" excessively, particularly when the excitation is near the window edges or in the presence of noise. This occurs in less than 5% of synthetic frames but is a notable limitation.
  • Sensitivity to Model Deviations: MED appears sensitive to deviations from the assumed signal model and the position of the signal within the analysis window.
  • Future Work: The authors plan to further combine MED and the hybrid method to leverage their individual strengths and achieve more robust estimates of the composite impulse response. They also aim to develop a full system implementation for MED-based analysis of continuous speech.

Summary updated

BibTeX
@inproceedings{malyska2006analysis,
  author = {Nicolas Malyska and Thomas F. Quatieri},
  title = {{Analysis of nonmodal phonation using minimum entropy deconvolution}},
  booktitle = {Interspeech},
  year = {2006},
}
Figure