A comprehensive 3d biomechanically-driven vocal tract model including inverse dynamics for speech research

Peter Anderson, Negar M. Harandi, Scott Moisik, Ian Stavness, Sidney FelsInterspeech 2015

TL;DR

This paper introduces a novel 3D biomechanical model of the human vocal tract, integrating soft palate, pharynx, and larynx with existing models of the face, jaw, skull, and hyoid. This comprehensive model enables both forward simulation of speech production and inverse dynamics for predicting muscle activations, paving the way for more realistic and physiologically grounded speech synthesis and research.

What Problem Does This Paper Address?

Understanding and simulating human speech production is a complex challenge. While previous research has developed models for individual components of the vocal tract (VT) like the face, tongue, or larynx, a unified, comprehensive biomechanical model that accurately captures the intricate interplay of these structures has been lacking. Existing speech synthesis models often rely on simplified 1D representations of the airway or manually tuned parameters, limiting their physiological realism and ability to investigate the underlying biomechanics of speech. This paper aims to bridge this gap by presenting a fully integrated 3D biomechanical VT model that can simulate speech production and predict muscle forces, offering a powerful tool for both forward and inverse dynamics in speech research.

Methods

The authors developed a comprehensive 3D biomechanical model of the upper airway using the Artisynth simulation toolkit. Key components include:

  • Finite Element (FE) Models: Deformable FE bodies represent the face, tongue, soft palate, pharynx, and larynx, capturing their complex geometries and material properties.
  • Rigid Bodies: The skull, jaw, and hyoid are modeled as rigid bodies, providing structural anchors and kinematic constraints.
  • Surface Meshes: Deformable skin meshes are used to represent the airway surface, dynamically deforming with the underlying articulators.
  • Point-to-Point Muscles: A large number of point-to-point muscles are implemented to simulate the forces that drive articulator movement. These muscles can be embedded within FE bodies or operate externally.
  • Coupling Mechanisms: Various coupling strategies are employed, including rigid-to-FE, FE-to-FE, and point-to-point muscle attachments, to create a unified and interconnected model.
  • Biomechanically Driven Acoustics: The model is coupled to a 1D acoustic synthesizer. The airway geometry, derived from the biomechanical simulation, is used to calculate area functions that shape the sound produced by a two-mass glottal model.
  • Inverse Dynamics: The framework supports inverse modeling, allowing the prediction of muscle activations required to achieve a target kinematic motion of the vocal tract.

Datasets

Not discussed in the paper. The authors mention using cone beam Computed Tomography (CT) data to design the pharynx geometry and anatomical references like Gray's Anatomy for model construction.

Key Insights and Ideas

  • Unified 3D Biomechanical Model: The integration of multiple vocal tract components (face, tongue, jaw, skull, hyoid, soft palate, pharynx, larynx) into a single, biomechanically driven 3D model is a significant advancement. This allows for the study of complex interactions between structures.
  • Forward and Inverse Dynamics: The model's ability to perform both forward (predicting motion from muscle forces) and inverse (predicting muscle forces from motion) simulations provides a versatile platform for speech research.
  • Airway Geometry vs. Acoustics: The paper highlights a critical challenge in 3D biomechanical VT modeling: the discrepancy between a geometrically precise airway mesh and a simplified, tube-like geometry that is easily processed by 1D acoustic models. This leads to differences in calculated area functions and, consequently, formant frequencies.
  • Physiological Realism: By directly simulating muscle activations and their resulting deformations, the model offers a more physiologically grounded approach to speech synthesis compared to purely geometric or acoustic models.

Key Results

  • Vowel Simulation: The model successfully simulated the articulation of vowels /a/, /i/, and /u/ by applying specific muscle activation patterns.
  • Acoustic Differences: The study demonstrated that different methods of calculating the airway's area function (e.g., CT-derived vs. exact airway with line-of-sight or contour CSA) lead to noticeable differences in the resulting formant frequencies, underscoring the importance of airway representation.
  • Inverse Prediction Fidelity: The inverse simulation successfully recreated the original muscle activations used in the forward simulation with good fidelity, validating the inverse dynamics approach.
  • Simulation Speed: Forward simulation of 0.4 seconds of speech took approximately 3.5 minutes, while inverse simulation took about 10.5 minutes on an 8-core PC.

Limitations and Future Work

  • Airway Mesh Remeshing: The authors suggest remeshing the soft palate to have a hexahedral mesh and the tongue to have finer elements for improved simulation quality.
  • Comprehensive Muscle Simulation: Future work aims to demonstrate comprehensive motions including pharyngeal and soft palate muscles.
  • Advanced Acoustic Modeling: The authors intend to incorporate features from the Birkholz model into their 1D acoustics model and explore 3D acoustic models.
  • Fluid-Driven Sound Source: A long-term goal is to generate sound by driving the larynx model with a fluid model, moving beyond mass-spring sound source models.
  • Airway Geometry Simplification: For 3D fluid simulations, the airway may need to be defined to exclude or simplify regions with minimal acoustic influence to reduce computational expense.
  • Airway Representation Challenges: The paper acknowledges the difficulty in obtaining a precise area function from a geometrically accurate 3D airway mesh, especially during airway closures.

Summary updated

BibTeX
@inproceedings{anderson2015comprehensive,
  author = {Peter Anderson and Negar M. Harandi and Scott Moisik and Ian Stavness and Sidney Fels},
  title = {{A comprehensive 3d biomechanically-driven vocal tract model including inverse dynamics for speech research}},
  booktitle = {Interspeech},
  year = {2015},
}
Figure