Toward aero-acoustical analysis of the sibilant /s/: an oral cavity modeling

Kazunori Nozaki, Youhei Ohnishi, Takeshi Suda, Shigeo Wada, Shinji ShimojoInterspeech 2010

TL;DR

This paper introduces a novel method for segmenting dynamic 4D-MRI data of the oral cavity, crucial for simulating the aero-acoustics of speech sounds like the sibilant /s/. By combining active contour models (Snakes) with optical flow, they achieve more accurate tracking of rapid tongue movements, enabling better numerical analysis of speech production.

What Problem Does This Paper Address?

Accurately modeling the production of speech sounds, particularly fricatives like the sibilant /s/, requires understanding the complex airflow dynamics within the oral cavity. This, in turn, necessitates precise geometric models of the vocal tract. Capturing the dynamic shape of the oral cavity, especially the rapid movements of the tongue during speech, is a significant challenge. Traditional methods for segmenting medical imaging data, like active contour models (Snakes), struggle with the fast deformations seen in 4D-MRI sequences. This limitation hinders the creation of accurate computational meshes needed for fluid dynamics simulations.

Methods

The authors propose a hybrid segmentation technique that combines the strengths of active contour models (specifically, Snakes) with optical flow (OF).

  • Snakes: This is an established method for segmenting images by deforming a curve (represented by control points) to fit object boundaries. It's chosen here because it maintains the topology of the segmented object, which is important for consistent mesh generation in simulations.
  • Optical Flow (OF): OF estimates the motion of objects between consecutive image frames. The authors use the pyramidal Lucas-Kanade (LK) method, a robust approach for tracking feature points.
  • The Hybrid Approach: The core innovation lies in using OF to guide and refine the Snakes segmentation. Specifically, the optical flow calculated from the previous time step's segmentation is used to "pre-condition" or initialize the contour for the current time step. This helps the Snakes algorithm adapt to rapid tongue movements that would otherwise cause segmentation errors or failures. The process involves:
  • Pre-conditioning 4D-MRI images.
  • Segmenting images using Snakes, with the initial contour for each frame informed by the optical flow from the previous frame.
  • Calculating the velocities of the control points (mesh nodes) based on the segmented contours across time steps.

Datasets

The study utilizes 4D-MRI data of the oral cavity while a subject pronounces the sequence "/u/-/s/-/u/-/i/".

  • Data Characteristics: The MRI data has a resolution of 128x128 pixels and a time-step resolution of 16 images per second. Seven longitudinal slice images were captured at each time step, resulting in 121 slice images per set.
  • Benchmark: The paper introduces a method for segmenting dynamic oral cavity shapes. While not a formal benchmark dataset release, the paper presents simplified oral cavity motion motifs (Figure 5) to validate their method, where the velocities of control points could be manually assigned.

Key Insights and Ideas

  • OF as a "predictor" for Snakes: The most significant insight is that optical flow can effectively predict and guide active contour models through rapid deformations, overcoming a key limitation of Snakes alone. This allows for more robust tracking of dynamic anatomical structures.
  • Bridging Imaging and Simulation: The method directly addresses the gap between dynamic imaging data and the requirements for computational fluid dynamics (CFD) and computational aero-acoustics (CAA) simulations. By accurately segmenting and obtaining mesh node velocities, it enables the creation of dynamic computational meshes.
  • Importance of Tongue Motion: The paper highlights that ignoring tongue motion leads to an incomplete understanding of sibilant production. Their method's ability to capture this motion is crucial for realistic aero-acoustical analysis.

Key Results

  • Improved Segmentation Accuracy: The proposed method, using Snakes with OF, demonstrated superior segmentation accuracy compared to using Snakes alone, particularly in following the quick motion of the tip of the tongue (Figure 6).
  • Accurate Velocity Calculation: The method successfully calculated the velocities of control points (mesh nodes) with high accuracy, especially when OF was incorporated. The results showed that OF was essential for accurately capturing the velocities of rapid object motion (Figure 7).
  • Fine Geometric Accuracy: The proposed method maintained fine accuracy in segmenting dynamic shapes, even when they changed significantly in height (Figure 8).

Limitations and Future Work

  • Manual Initialization: The segmentation process still requires manual preparation of an initial contour (IC) on the first image.
  • Window Size Determination: The window size for the optical flow calculation was determined experimentally and may need further analysis to be suitable for a wider range of movements.
  • Future Work:
    • Segmenting all slices of the 4D-MRI data to obtain full 3D dynamic models.
  • Analyzing more cases to determine optimal OF window sizes for various movements.
  • Superimposing tooth information (using complementary registration methods) to create more complete vocal tract models for sibilant /s/ and adjacent phonemes.
  • Performing numerical simulations using the ALE method with the obtained dynamic oral cavity models.

Summary updated

BibTeX
@inproceedings{nozaki2010toward,
  author = {Kazunori Nozaki and Youhei Ohnishi and Takeshi Suda and Shigeo Wada and Shinji Shimojo},
  title = {{Toward aero-acoustical analysis of the sibilant /s/: an oral cavity modeling}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure