Time-varying autoregressive tests for multiscale speech analysis

Daniel Rudoy, Thomas F. Quatieri, Patrick J. WolfeInterspeech 2009

TL;DR

This paper introduces a novel hypothesis testing framework based on time-varying autoregressive (TVAR) models to detect nonstationarity in speech signals. This approach allows for more accurate segmentation of speech at both broad (segmental) and fine (sub-segmental) scales, leading to improved analysis of vocal tract dynamics.

What Problem Does This Paper Address?

Traditional speech analysis often relies on breaking down audio into short, fixed-time segments (e.g., 15-30 ms). While this assumes local stationarity, the reality is that speech signals are continuously evolving due to the dynamic nature of the vocal tract. This paper addresses the limitation of fixed-segment analysis by proposing a method that can adaptively identify regions where the speech signal's spectral characteristics are changing, and conversely, regions where they remain stable. This is crucial for tasks like accurate speech segmentation and understanding subtle vocal events like the glottal closed phase.

Methods

The core of this work is the development of a Generalized Likelihood Ratio Test (GLRT) tailored for Time-Varying Autoregressive (TVAR) models.

  • TVAR Model: Instead of assuming constant autoregressive (AR) coefficients within a segment (standard AR model), TVAR models allow these coefficients to evolve over time. This is achieved by representing the time-varying coefficients as a linear combination of predefined basis functions (like Legendre or Fourier polynomials). This more accurately captures the smooth, continuous changes in the vocal tract.
  • GLRT for Hypothesis Testing: The paper formulates a hypothesis test:
  • Null Hypothesis (H₀): The speech segment is well-modeled by a standard AR process (i.e., coefficients are constant).
  • Alternative Hypothesis (H₁): The speech segment is better modeled by a TVAR process (i.e., coefficients are changing). The GLRT statistic is used to decide between these hypotheses. The authors highlight that using TVAR models with a GLRT provides estimates of the time-varying coefficients, which are valuable for analysis.

  • Adaptive Segmentation: The GLRT is then integrated into an adaptive Short-Time Fourier Transform (STFT) analysis. Instead of fixed windows, neighboring segments are merged if the GLRT indicates no significant change (i.e., H₀ is accepted). This results in longer windows for stable speech regions (improving spectral resolution) and shorter windows for changing regions (preserving temporal detail).

Datasets

The paper evaluates its methods on:

  • Synthetic Signals: Used to characterize the GLRT's detection performance across varying frequency jumps and signal lengths.
  • Whispered Speech Recordings: Two specific utterances, "[a aI]" (vowel followed by diphthong) and "[i t]" (steady-state vowel followed by a plosive), were used to demonstrate segmental analysis.
  • Voiced Vowel Recordings: Used for sub-segmental analysis of the glottal closed phase. These were synchronously recorded with Electroglottograph (EGG) signals, which served as "ground truth" for identifying the closed phase.

Key Insights and Ideas

  • Speech is Inherently Nonstationary: The paper strongly advocates for moving beyond fixed-segment analysis to account for the continuous evolution of vocal tract dynamics.
  • TVAR Models are a Better Fit: TVAR models, by allowing coefficients to change, provide a more realistic representation of speech signals than traditional AR models.
  • GLRT as a Powerful Detector: The GLRT framework is shown to be effective in detecting spectral changes, even at sub-segmental scales where data is limited.
  • Adaptive Windowing for Better Analysis: Merging segments based on the GLRT's nonstationarity detection leads to an adaptive analysis that optimizes spectral resolution and temporal precision simultaneously.
  • Sub-Segmental Analysis Potential: The ability to detect subtle changes, like the glottal closed phase, opens doors for more refined speech analysis techniques.

Key Results

  • Synthetic Data: ROC curves demonstrate that the GLRT's detection performance improves with larger frequency jumps and longer signal durations.
  • Segmental Analysis (Whispered Speech):
    • The adaptive STFT successfully identified regions of formant motion (diphthong transitions) and stable formants (vowel).
  • For "[a aI]", stable vowel regions were analyzed with longer windows, while formant transitions were analyzed separately with shorter windows.
  • For "[i t]", the vowel /i/ received a long window, the plosive /t/ a short one, and even the silence and aspiration phases were isolated.
  • Sub-Segmental Analysis (Glottal Closed Phase):
    • The GLRT was effective in identifying the end of the glottal closed phase by detecting spectral changes associated with the onset of the open phase.
  • The detected end of the closed phase closely coincided with dips in the EGG derivative.
  • Root Mean Square Errors (RMSE) between detected closure end and EGG dips were small (e.g., 0.69 for /a/, 1.31 for /e/), indicating promising accuracy.

Limitations and Future Work

  • Limitations:
    • The paper notes that increasing the number of basis functions (q) to better model coefficient trajectories can sometimes decrease test sensitivity due to increased degrees of freedom.
  • The performance of the GLRT is sensitive to the choice of AR model order (p) and the number of basis functions (q).
  • The current work focuses on whispered speech for segmental analysis; extending to voiced speech is mentioned as future work.
  • Future Work:
    • Applying the GLRT to detect vocal tract changes in voiced utterances.
  • Further investigation into using the GLRT for identifying glottal closure instants.
  • Exploring the use of the GLRT for speaker identification and inverse filtering.

Summary updated

BibTeX
@inproceedings{rudoy2009time,
  author = {Daniel Rudoy and Thomas F. Quatieri and Patrick J. Wolfe},
  title = {{Time-varying autoregressive tests for multiscale speech analysis}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure