Estimation of vocal tract shapes from speech sounds with a physiological articulatory model

Jianwu Dang, Jianwu Dang 0001, Kiyoshi HondaJ. Phonetics 2002 · DOI

TL;DR

This paper presents a novel method for estimating vocal tract shapes from speech sounds by employing a sophisticated physiological articulatory model. This approach leverages anatomical and dynamic constraints to improve the accuracy and realism of vocal tract reconstruction, offering a more robust solution to the long-standing inverse problem in speech research.

What Problem Does This Paper Address?

Estimating the shape of the vocal tract from speech sounds is a fundamental challenge in speech research. This is known as the "inverse problem" because a single speech sound can be produced by many different vocal tract configurations (the "one-to-many" problem), and the relationship between articulatory movements and acoustic output is inherently nonlinear. Previous methods often relied on idealized acoustic tube models or statistical articulatory models, which may not fully capture the physiological complexities of human speech production. This can lead to estimations that are not anatomically plausible or dynamically realistic. The authors aim to overcome these limitations by using a detailed, physiologically constrained 3D articulatory model.

Methods

The core of this work is a 3D physiological articulatory model built from volumetric MRI data of a male speaker. This model incorporates:

  • Morphological and Dynamic Constraints: Unlike simpler models, this one includes realistic anatomical shapes and simulates the dynamic interplay of muscles.
  • Dynamic Muscle Workspace: To account for how muscle orientations change during articulation, a dynamic muscle workspace is introduced. This allows for more flexible control of articulatory movements.
  • Multipoint Control Strategy: The model uses three control points (tongue tip, tongue dorsum, and jaw) to manage articulatory movements, offering finer control over different parts of the vocal tract.
  • Target-Based Control Strategy: The system aims to achieve specific articulatory targets by generating muscle activation patterns. This is achieved by mapping desired positions in the articulatory space to muscle force vectors within a defined workspace.
  • Acoustic-to-Articulatory Mapping: The paper proposes an "analysis-by-synthesis" (AbS) approach. This involves generating synthetic speech from the articulatory model and then iteratively adjusting the articulatory parameters to minimize the difference between the acoustic features of the synthetic and real speech.
  • Articulatory Constraint: A key innovation is the use of an articulatory constraint derived from the correlation between tongue dorsum position and the F1-F2 formant difference. This constraint helps to improve the accuracy of the inverse estimation, particularly for vowels where lip and tongue movements can compensate for each other.

Datasets

  • Volumetric MRI Data: Used to construct the 3D physiological articulatory model of a male speaker.
  • X-ray Microbeam Data: Collected from six male Japanese speakers, this data was used to establish the correlation between tongue dorsum position and formant frequencies, forming the basis for the articulatory constraint.
  • Japanese Vowel Sequences: Used for evaluating the estimation method. Both synthetic sounds (generated from known articulatory targets) and real speech sounds were utilized.

Key Insights and Ideas

  • Physiological Realism Matters: Incorporating detailed anatomical and dynamic constraints from a physiological model significantly improves the plausibility and accuracy of vocal tract shape estimation compared to purely acoustic or statistical approaches.
  • Dynamic Muscle Workspaces Enhance Control: The concept of dynamic muscle workspaces allows the model to adapt to changing muscle orientations during articulation, leading to more realistic movement simulation.
  • Formant-Tongue Correlation as a Powerful Constraint: The strong correlation between the F1-F2 formant difference and tongue dorsum position provides a valuable, physiologically grounded constraint for the inverse estimation problem. This helps to resolve ambiguities and improve accuracy.
  • AbS with Physiological Constraints is Effective: The analysis-by-synthesis method, when integrated with the physiological constraints of the articulatory model, offers a robust framework for estimating vocal tract shapes from speech.

Key Results

  • Vocal Tract Shape Estimation Error: For the target speaker, the average error in estimating vocal tract shapes was 0.16 cm.
  • Formant Estimation Error: The average error for the four lower formants was 1.8%.
  • Articulatory Constraint Improvement: Implementing the articulatory constraint led to improved estimation accuracy, particularly for vowels like /u/ where lip and tongue movements can be compensatory. For example, the average error for vowel sequences decreased from about 2.5% to 1.8%.
  • Correlation Strength: The correlation coefficient between the horizontal position of the tongue dorsum and the F1-F2 difference was 0.956 for the target speaker, indicating a strong relationship.

Limitations and Future Work

  • Consonant Estimation: The current method is primarily evaluated for vowel sequences. Extending the method to accurately estimate vocal tract shapes for speech sounds with consonants is identified as a crucial area for future work.
  • Lip/Tongue Compensation: While the articulatory constraint helps, the paper notes that lip/tongue compensation for certain sounds (like /u/) can still affect formant frequencies and other acoustic characteristics (e.g., spectral tilt). More sophisticated constraints might be needed to fully address these complex interactions.
  • Full 3D Model: The current tongue model is a partial 3D sagittal layer. Developing a full 3D model could further enhance realism.
  • Nasal Sounds: The velum was not used as an articulatory parameter because nasal sounds were not addressed in this study. Future work could incorporate velum dynamics for a more complete model.

Summary updated

BibTeX
@article{dang2002estimation,
  author = {Jianwu Dang and Jianwu Dang and Kiyoshi Honda},
  title = {{Estimation of vocal tract shapes from speech sounds with a physiological articulatory model}},
  journal = {J. Phonetics},
  year = {2002},
  doi = {10.1006/jpho.2002.0167},
}
Figure