Abstract
This paper presents a novel algorithm designed to overcome the limitations of current speech imaging techniques. By integrating the high spatial resolution of static 3D MRI with the high temporal resolution of real-time 2D MRI (rtMRI), the researchers developed a pipeline to generate dynamic 3D models of the vocal tract. Using a non-rigid image transformation approach based on the "Demons algorithm," the method "inflates" 2D video sequences using 3D anatomical targets. Results demonstrate that the synthesized sequences maintain anatomical richness while capturing natural fluid motion, offering a significant improvement over traditional 2D rtMRI.
Introduction
Understanding the mechanics of speech production requires high-fidelity visualization of the vocal tract in motion. However, researchers face a trade-off between spatial and temporal resolution in Magnetic Resonance Imaging (MRI). Static 3D MRI provides detailed volumetric data but requires subjects to remain still, which is unsuitable for capturing natural speech. Conversely, real-time 2D MRI (rtMRI) captures motion at high frame rates but is limited to a single plane and often suffers from noise, low contrast, and artifacts. This study proposes a semi-automated procedure to bridge this gap by synthesizing 3D dynamic sequences from 2D recordings and 3D static targets.
Methods
The proposed pipeline utilizes several advanced image processing techniques to transform 2D temporal data into 3D volumes:
- Non-Rigid Image Transformation: The "Demons algorithm" for image registration calculates displacement fields for every pixel, allowing the model to adapt to complex anatomical variations between speakers and articulatory positions.
- Double 2D Mapping: Static 3D targets for consonants (C) and vowels (V) are mapped to the corresponding frames in a 2D rtMRI CV transition.
- 3D Synthesis: Deformation fields from the mid-sagittal plane are propagated to synthesize neighboring sagittal slices, extending the 2D data into the third dimension.
- Advanced Denoising: To mitigate rtMRI artifacts, the team employed Canny edge detection to define boundaries, followed by edge-aware Gaussian filtering to maintain sharpness.
Key Results
The algorithm was tested on data from two speakers across 12 consonant-vowel (CV) syllables. Qualitative validation by linguists confirmed that the synthesized sequences accurately represented the intended speech dynamics.
Key findings include:
- Cross-Speaker Robustness: The method successfully mapped one speaker's 2D dynamics onto another's 3D anatomy, suggesting the algorithm is robust to different vocal tract shapes.
- Enhanced Resolution: The synthesized mid-sagittal slices exhibited significantly higher contrast and resolution than the raw 2D data.
- Anatomical Plausibility: The generation of non-mid-sagittal frames resulted in anatomically plausible 3D volumes, despite the lack of direct 2D references for those planes.
Discussion and Limitations
While successful, the "Demons algorithm" occasionally struggled with physical contact points, such as lip closure for /p/ or tongue-palate contact for /t/, leading to unnatural gaps in the deformation fields. Additionally, some transitions exhibited minor "jerks" at the points where forward and backward propagated sequences were joined (the "gluing point"). Future work will focus on full automation to enable the enrichment of large-scale 2D MRI databases into high-fidelity 3D models.
Conclusion
This research provides a viable pathway for creating high-quality dynamic 3D models of the vocal tract. By combining the strengths of 2D and 3D MRI, the method offers a powerful tool for speech science, enabling more detailed study of articulatory dynamics without sacrificing temporal accuracy.
Summary updated