Speech is an inherently three-dimensional phenomenon. To produce speech, the human vocal tract acts as a dynamic acoustic filter, reshaping an airway via the coordinated movements of the lips, jaw, velum, and tongue. The tongue itself is a muscular hydrostat—an incompressible structure that deforms in complex, multi-axial ways, much like an octopus tentacle or an elephant trunk.
Yet, for decades, the empirical foundation of speech science has rested on a two-dimensional simplification: the midsagittal plane. Because capturing high-speed volumetric motion is computationally and physically challenging, standard real-time Magnetic Resonance Imaging (rtMRI), ultrasound, and electromagnetic tracking have traditionally captured only a single slice along the vertical centerline of the face.
As Asterios Toutios notes in Advances in real-time magnetic resonance imaging of the vocal tract for speech science and technology research, analyzing speech through this single slice is akin to trying to understand a complex engine by looking at a single piston. This planar imaging constraint creates a fundamental dimensionality gap between our observations and the physical reality of speech.
The Hidden Mechanics of the Third Dimension
Relying on a single 2D slice requires a major assumption: that the vocal tract is laterally symmetrical and that midline movements represent the entire airway. In reality, many of the critical articulatory maneuvers required for human language occur entirely off-axis.
Consider the production of sibilant fricatives like /s/. To create the high-velocity jet of air necessary for a hiss, the tongue must form a tight, central groove along its length. In a standard 2D sagittal slice, this groove is completely invisible; the imaging plane merely cuts through a single point of the curve, masking the lateral curling and deep deformations highlighted by Yinghua Zhu et al..
Standard 2D Sagittal Slice True 3D Cross-Section (Coronal View)
___________ ___________
/ Tongue \ / _ _ \
/ Line \ / / \___/ \ \
[====== X ======] [=== Groove ===]
^ ^
Plane of view Central channel
(Groove is invisible) visible off-axis
This limitation is equally problematic when modeling liquid consonants, such as the distinction between a lateral [l] and a flap [ɾ]. As Yoon-Jeong Lee et al. demonstrate, these sounds rely heavily on lateral tongue-to-palate contact and the three-dimensional management of airflow around the sides of the tongue—dynamics that escape 2D midline profiles entirely. Beyond the tongue, critical volume changes also occur through the expansion and contraction of the lateral pharyngeal walls, which remain hidden from a sagittal perspective.
When we ignore these transverse dimensions, our downstream models break down. In A Multimodal Real-Time MRI Articulatory Corpus of French for Speech Research, Ioannis Douros explains that omitting the transverse width of the vocal tract introduces severe errors when predicting acoustic formants and spectral peaks. Because acoustic resonance is determined by the true volumetric cross-sectional area function of the tube, a 2D slice simply lacks the geometric information required for high-fidelity speech synthesis or accurate acoustic-to-articulatory inversion.
Strategies for Volumetric Reconstruction
To overcome planar constraints without sacrificing the temporal resolution needed to track fluid speech, researchers have developed several distinct strategies to capture or infer the missing third dimension.
Interleaved Multi-Planar Imaging
Rather than attempting full 3D volumetric imaging—which can be too slow to capture rapid speech gestures—one approach rapidly samples multiple 2D planes. Yoon-Chul Kim et al. developed a real-time MRI technique that rapidly switches (interleaves) between midsagittal, coronal, and axial views every few milliseconds. By stitching these orthogonal perspectives together, researchers can simultaneously monitor midline tracking alongside lateral pharyngeal expansion and tongue grooving.
Accelerated Volumetric Acquisition
An alternative is to push the boundaries of MRI physics to acquire true 3D volumes over time (4D imaging). Yinghua Zhu et al. achieved this by combining a "stack-of-spirals" k-space sampling pattern with a constrained mathematical reconstruction model that mathematically separates spatial and temporal data. This approach allowed them to capture full 3D vocal tract shapes during continuous speech at 8.6 frames per second, making complex, rapid 3D tongue deformations directly visible.
Cross-Modal Shape Inflation
Another prominent strategy uses data-driven modeling to "inflate" high-speed 2D data into 3D shapes. Ioannis Douros developed an algorithm that pairs the high temporal resolution of dynamic 2D rtMRI with the high spatial resolution of static 3D MRI scans (Towards a Method of Dynamic Vocal Tract Shapes Generation by Combining Static 3D and Dynamic 2D MRI Speech Data). By using the static 3D shapes as anatomical targets, the 2D video sequences are effectively mapped into a dynamic 3D space.
Similarly, in ultrasound imaging—which traditionally suffers from the same sagittal slice limitations detailed by Kristofer Bouchard et al.—Yin Yang 0002 et al. built a framework that extracts 2D tongue contours from ultrasound frames and matches them against a pre-computed database of physics-based 3D tongue shapes to synthesize realistic 3D motion in near real-time.
Parasagittal Physical Tracking
The push for three-dimensional fidelity has also altered physical motion-capture paradigms like Electromagnetic Articulography (EMA). Historically, researchers placed receiver coils exclusively along the tongue's midline. To test whether this omission mattered, Jun Wang 0037 evaluated the explicit contribution of lateral sensors in Contribution of tongue lateral to consonant production.
While midline sensors capture a high degree of redundant information for simpler sounds, Salvador Medina et al. demonstrated that adding just two parasagittal sensors to the sides of the tongue uncovers a complex world of rolling, twisting, and cupping. This extra spatial information proves vital for capturing the true physical constraints of the muscular hydrostat during natural, unconstrained articulation.
Key Foundational Work
Speaker adaptation of a three-dimensional tongue model (2004) — contributed 1 concept elements to this direction.
Estimation of vocal tract shapes from speech sounds with a physiological articulatory model (2002) — contributed 1 concept elements to this direction.