Speech Synthesis via Vocal Tract Kinematics

33 concepts30 papers2 proposals

Traditional text-to-speech and speech coding systems operate primarily in the acoustic domain. They treat speech as a sequence of abstract spectral coefficients or raw waveforms. While computationally convenient, this "black box" approach ignores the physical reality of how speech is produced: as a dynamic fluid-structure interaction inside the human vocal tract.

Articulatory speech synthesis shifts the paradigm by modeling the physical engine itself. The human speech production system can be conceptualized as a source-filter model. The source—typically the glottal flow through the vocal cords—generates a rich harmonic signal. As shown by Baris Bozkurt et al. in A method for glottal formant frequency estimation and Thomas Drugman et al. in Complex cepstrum-based decomposition of speech for glottal source estimation, isolating this glottal source from the vocal tract filter is essential for high-fidelity voice analysis. This source signal then passes through the vocal tract, which acts as a continuously morphing acoustic filter, shaping the sound into recognizable phonemes.

Parameterizing the Vocal Tract Geometry

How do we mathematically represent a shape as complex as the human mouth, tongue, and throat? Instead of tracking every individual muscle fiber, researchers use low-dimensional geometric models to capture the principal degrees of freedom.

A prominent framework for this is the Maeda articulatory model. As leveraged by Benjamin Elie et al. in Articulatory Configurations across Genders and Periods in French Radio and TV archives, the Maeda model reduces the entire vocal tract shape to a small set of highly interpretable parameters: jaw position, tongue dorsum position and height, tongue tip position, lower lip height, lip protrusion, and larynx height. Sorin Dusan demonstrates in Vocal tract length during speech production that these parameters are highly coordinated; for instance, changes in vocal tract length dynamically correlate with lip protrusion and larynx height during natural speech.

By defining speech via these kinematic parameters, we enforce a powerful physical prior: the model can only generate shapes that a human vocal tract can actually physically achieve.

The Forward Mapping: From Kinematics to Acoustics

Once we have a parameterized vocal tract shape, how does it become sound? The forward mapping translates geometric configurations into acoustic area functions, which dictate how sound waves propagate and resonate.

In Evoc-Learn — High quality simulation of early vocal learning, Yi Xu et al. implement a 3D geometric model that simulates aeroacoustic properties and articulatory dynamics, treating speech gestures as discrete target movements. This physical simulation guarantees that the resulting audio possesses a coherent harmonic structure. In fact, Ryan J. Cassidy et al. show in Auditory Display of Hyperspectral Colon Tissue Images Using Vocal Synthesis Models that the human auditory system is uniquely optimized to categorize and remember these physically constrained harmonic structures compared to abstract acoustic noise.

To ground these models in real biology, researchers use medical imaging. Yves Laprie et al. demonstrate in Articulatory copy synthesis from cine x-ray films that acoustic simulations can be driven directly by vocal tract area functions derived from cine X-ray films. Similarly, Jacqueline Vaissière outlines in From X-ray or MRU data to sounds through articulatory synthesis: towards an integrated view of the speech communication process how bridging the gap between X-ray/MRI imaging and acoustic output provides a unified framework for understanding speaker-specific variations.

The Inverse Problem: Acoustic-to-Articulatory Inversion

If the forward problem is mapping movement to sound, the inverse problem is mapping sound back to movement. This is how humans learn to speak—by listening to an acoustic target and adjusting their internal articulatory parameters to match it.

Santitham Prom-on et al. explore this in Training an articulatory synthesizer with continuous acoustic data, demonstrating that an articulatory synthesizer can learn to mimic human speech acoustics with high formant correlations by optimizing its internal trajectories. This developmental process is further mirrored in computational models of infant-adult interaction, such as the work by Mark Huckvale et al. in Learning to imitate adult speech with the KLAIR virtual infant.

Beyond synthesis, tracking these physical movements via ultrasound imaging allows researchers to study natural speech variations. As shown by Jeffrey Berry et al. in Articulatory reduction in Mandarin Chinese words, repeated words exhibit a physically reduced tongue movement range and faster execution, providing an articulatory explanation for acoustic speech reduction.

Modern deep learning has dramatically accelerated this inversion. In Ultra2Speech - A Deep Learning Framework for Formant Frequency Estimation and Tracking from Ultrasound Tongue Images, Pramit Saha et al. use a deep neural network to map non-invasive ultrasound tongue images directly to acoustic formants with near-perfect regression accuracy, enabling high-fidelity speech reconstruction from physical movement alone.

Why the Articulatory Prior Changes Speech Technology

Why go through the trouble of modeling physical kinematics instead of using standard neural audio generation? The answer lies in interpretability, generalization, and physical realism.

Extrapolation and Control

Traditional statistical speech synthesis is bound by its training data. However, by mapping acoustics to physical movements, systems can generate entirely unseen sounds. Zhen-Hua Ling et al. demonstrate this in Articulatory control of HMM-based parametric speech synthesis driven by phonetic knowledge and Vowel Creation by Articulatory Control in HMM-based Parametric Speech Synthesis, where they successfully synthesize completely new vowels by manipulating virtual physical parameters rather than abstract mathematical coefficients.

High-Fidelity Speech Coding

Instead of compressing audio into abstract neural tokens, frameworks like SPARC, introduced by Cheol Jun Cho et al. in Coding Speech Through Vocal Tract Kinematics, represent speech via vocal tract movements and source characteristics. This provides an inherently controllable and speaker-agnostic representation for speech transmission.

Clinical Applications and Silent Speech

Understanding the physical constraints of speech is vital for assistive technologies. Beiming Cao et al. investigate this in Investigating Speech Reconstruction for Laryngectomees for Silent Speech Interfaces, mapping tongue and lip positions to restore speech for laryngectomees.

Furthermore, when generating synthetic data for clinical tasks like stuttering detection, traditional text-to-speech lacks physical realism. As argued by Xuanru Zhou et al. in Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection and Jinming Zhang et al. in Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection, embedding articulatory priors and vocal tract kinematics into synthesis pipelines yields far more authentic simulated dysfluencies than manipulating raw waveforms.

Finally, understanding these physical constraints helps explain how individuals adapt to limitations. As shown by Jörg Metzner et al. in Substitute sounds for ventriloquism and speech disorders, when the physical vocal tract is constrained (as in ventriloquism or speech disorders), speakers can find alternative articulatory configurations to reproduce the necessary acoustic spectral information.

Key Foundational Work

From X-ray or MRU data to sounds through articulatory synthesis: towards an integrated view of the speech communication process (2004) — contributed 2 concept elements to this direction.
Substitute sounds for ventriloquism and speech disorders (2006) — contributed 1 concept elements to this direction.
A method for glottal formant frequency estimation (2004) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_maeda_articulatory_model_ccb41fab,
  title = {Maeda articulatory model},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/maeda-articulatory-model/rdr_ccb41fabfe8d4aa7970dcc20e1c9dfb5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_maeda_articulatory_model_ccb41fab_v1,
  title = {Maeda articulatory model},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/maeda-articulatory-model/rdr_ccb41fabfe8d4aa7970dcc20e1c9dfb5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake