Detection of transition segments in VCV utterances for estimation of the place of closure of oral stops for speech training

J. Jagbandhu, K. S. Nataraj, Prem C. PandeyInterspeech 2012

TL;DR

This paper introduces a novel method for automatically detecting the crucial transition periods in speech, specifically within vowel-consonant-vowel (VCV) utterances. By accurately pinpointing these transitions, the researchers enable more precise estimation of where in the mouth a speaker forms stop consonants, a key feature for developing better speech training tools for individuals with hearing impairments.

What Problem Does This Paper Address?

Accurate speech production relies on precise articulation, especially for stop consonants like /p/, /t/, and /k/. For speech training aids, particularly those designed for the hearing impaired, providing visual feedback on articulatory efforts is crucial. While estimating the shape of the vocal tract during vowel sounds is relatively straightforward using techniques like Linear Predictive Coding (LPC), this method falters during the brief, low-energy closure phase of stop consonants. Previous work has shown that the vocal tract shape during these closures can be effectively interpolated by modeling the articulatory changes during the transition segments before and after the closure. However, the accuracy of this interpolation hinges on precisely identifying the start and end of these transition segments. Existing methods for detecting these transitions are either manual, require empirical tuning that is difficult for individuals with speech impairments, or lack the precision needed for accurate boundary marking. This paper aims to bridge this gap by developing an automated, robust method for detecting these critical transition segments.

Methods

The core of this paper lies in a new technique for automatically detecting the VC (vowel-to-consonant) and CV (consonant-to-vowel) transition segments in VCV utterances. The authors leverage the vocal tract area function, estimated using LPC analysis. Instead of directly using the area values, they focus on the rate of change of these area values over time.

Initially, they explored a combined rate of change measure based on the slopes of the first six vocal tract sections. However, this proved inconsistent. The breakthrough came with using the time-slope of a bivariate linear approximation fitted to the vocal tract area function over a moving window of frames. This coefficient, denoted as c₁, effectively captures the rate of change of the vocal tract shape along the time axis.

  • Bivariate Linear Approximation: A moving 7-point linear regression is applied to the vocal tract area function g(x, y) (where x is time and y is vocal tract section) to estimate coefficients C₀, C₁, and C₂.
  • c₁ as Rate of Change: The coefficient c₁ from this approximation is identified as a reliable indicator of articulatory change. During steady vowels, c₁ is small. During VC transitions, it becomes negative, peaking near closure. During CV transitions, it turns positive, peaking at the start of the transition.
  • Transition Boundary Detection: The start and end of the stop closure are initially estimated using short-time average magnitude. Then, the detected c₁ peaks and valleys are used to refine these boundaries. Specific rules are implemented to handle potential issues like aspiration in unvoiced stops, ensuring robustness.

Datasets

The study utilized VCV utterances of the type /aCa/ (where 'C' represents stop consonants /p, b, t, d, k, g/). These utterances were recorded from two male and two female speakers. The accuracy of the automated transition markings was evaluated by comparing them against manual markings obtained by visual inspection of waveforms and spectrograms using the PRAAT software. The paper also references the XRMB database [12] for comparison of estimated place of closure with direct X-ray imaging.

Key Insights and Ideas

  • Rate of Change as a Key Indicator: The most significant insight is that the rate of change of the vocal tract area function, rather than the absolute area values, provides a robust signal for identifying articulatory transitions.
  • c₁ Coefficient's Predictive Power: The c₁ coefficient from a bivariate linear approximation elegantly captures the dynamics of vocal tract shape changes, making it a powerful tool for marking the start and end of VC and CV transitions.
  • Automated Detection for Improved Training: By automating the detection of these transition segments, the method paves the way for more accurate and consistent visual feedback in speech training aids, especially for individuals who may struggle with manual calibration or have atypical speech patterns.
  • Consistency Across Speakers: The observed patterns in c₁ across different speakers suggest the universality of this approach for detecting transitions in VCV utterances.

Key Results

The automated transition segment detection method demonstrated a good match with manually marked boundaries.

  • Error Analysis: The errors in marking the four transition points (VC start, VC end, CV start, CV end) were generally small, with a mean error of 8.1 ms for VC start, 8.3 ms for VC end, 3.9 ms for CV start, and 7.7 ms for CV end. The standard deviations were also low (e.g., 2.7 ms for CV end), indicating high consistency.
  • Place of Closure Estimation: When these automatically detected transition segments were used for estimating the place of maximum constriction during stop closures, the results showed good agreement with values obtained from MRI and X-ray imaging. For example, the largest constrictions for /p/, /t/, and /k/ were found at normalized distances of 1, 0.8, and 0.6, respectively, closely matching established values.
  • Qualitative Results: Figure 3 visually demonstrates the effectiveness of the method, showing areagrams (visual representations of vocal tract area over time and position) where the detected transitions are clearly indicated, and the interpolated shapes during closure are well-represented.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that one large error of 30 ms occurred in marking the VC transition start for a specific utterance, though it didn't significantly impact the overall modeling due to the utterance's longer transition.
  • The current technique is primarily evaluated on isolated VCV utterances.
  • The authors suggest that further refinement might be needed for application to utterances from children with speech impairments, who may exhibit different articulatory dynamics.
  • Future Work:
    • The authors propose further evaluation of the technique on VCV utterances from speakers of different languages and accents.
  • They also suggest refining the method for applications involving children with speech impairments.
  • Beyond speech training, the technique is deemed useful for other non-real-time speech processing applications that require accurate transition marking.

Summary updated

BibTeX
@inproceedings{jagbandhu2012detection,
  author = {J. Jagbandhu and K. S. Nataraj and Prem C. Pandey},
  title = {{Detection of transition segments in VCV utterances for estimation of the place of closure of oral stops for speech training}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure