TL;DR
This paper introduces a novel method for visualizing and analyzing tongue movement during speech and swallowing using ultrasound. By employing kriging to extrapolate and resample tongue surface contours, the authors overcome challenges like varying contour lengths and spatial shifts, enabling more robust statistical analysis and detailed spatiotemporal surface visualization.
What Problem Does This Paper Address?
Understanding human tongue motion is crucial for fields like speech pathology and biomechanics. Researchers often use 2D ultrasound to capture tongue surface movements at high frame rates. However, analyzing this data quantitatively is challenging due to several factors:
- Inconsistent Contour Lengths: The tongue's length can vary naturally during speech or swallowing, and even across repetitions of the same utterance. This makes direct averaging or comparison difficult.
- Spatial Variability: Ultrasound transducer positioning can shift between recordings, leading to arbitrary spatial shifts and potential data loss in the extracted contours.
- Uneven Sampling: The contour extraction process can result in varying sampling densities along the tongue surface, further complicating analysis.
These issues prevent straightforward averaging and comparative analysis of tongue shapes, hindering the ability to draw statistically significant conclusions about tongue dynamics.
Methods
The core innovation of this paper lies in its pre-processing approach using kriging. Kriging is a geostatistical technique that estimates values at unobserved locations based on observed data and their spatial correlation. Here's how it's applied:
- Contour Extraction: 2D sagittal ultrasound images are acquired, and the tongue surface contours are automatically extracted and tracked using deformable contours.
- Kriging for Extrapolation and Resampling:
- Extrapolation: Kriging is used to extend the contours beyond their original sampled points. This is crucial for making contours of different lengths comparable. Unlike traditional polynomial fitting, kriging is less prone to wild oscillations, especially at the contour's edges.
- Resampling: The extrapolated contours are then resampled onto a regular spatial grid. This ensures that all contours have the same number of points and are aligned spatially.
- Smoothing: The paper utilizes a spatial smoothing version of kriging, which accounts for noise in the ultrasound data, making the resulting continuous function more robust.
- Spatiotemporal Surface Generation: The processed, equally-lengthed, and regularly sampled contours from a sequence of images are stacked to form a spatiotemporal surface. This surface visually represents the tongue's movement over time in 3D.
- "SURFACES" Software: A dedicated user interface, "SURFACES," was developed in MATLAB to implement the kriging algorithm and visualize these spatiotemporal surfaces. It also supports averaging multiple repetitions to create a stable average surface and comparing different surfaces.
Datasets
The paper describes data acquisition from ultrasound images of the mid-sagittal section of the tongue. Subjects were asked to repeat utterances multiple times (typically 5-7 repetitions) to account for intra-subject variability. The audio data was also recorded but not directly used in this paper. The paper does not introduce a new benchmark dataset but rather uses data collected for specific studies:
- Upright vs. Supine Speech: Comparing tongue shapes during speech in upright and supine positions.
- Swallowing Study: Analyzing tongue movement during swallowing with simulated anterior open bite.
Key Insights and Ideas
- Kriging as a Superior Extrapolation Tool: The paper convincingly demonstrates that kriging outperforms traditional methods like cubic splines for extrapolating tongue contours. Kriging's ability to handle spatial correlations and its asymptotic properties lead to smoother, more realistic extrapolations, avoiding the "wild oscillations" seen with splines.
- Enabling Robust Statistical Analysis: By standardizing contour lengths and spatial sampling, kriging unlocks the potential for meaningful statistical analysis (e.g., Principal Component Analysis) of tongue shapes across different utterances, subjects, and conditions.
- Spatiotemporal Surfaces for Intuitive Visualization: The "SURFACES" software provides a powerful way to visualize complex tongue movements as 3D surfaces over time. This allows for both qualitative assessment of speech and swallowing dynamics and serves as a foundation for quantitative comparisons.
- Accounting for Noise: The use of the smoothing version of kriging, which incorporates a noise covariance matrix, is a practical improvement for real-world ultrasound data, which is inherently noisy.
Key Results
The paper primarily presents a methodology and a software tool. The key results are qualitative and demonstrated through visualizations:
- Improved Extrapolation: Figure 2 clearly shows that kriging produces a much smoother and more biologically plausible extrapolation of the tongue contour compared to cubic splines, particularly at the back of the tongue.
- Spatiotemporal Surface Visualization: Figures 3 and 4 showcase the generated spatiotemporal surfaces for the word "golly" and a swallow. These visualizations effectively illustrate tongue shape changes over time, revealing subtle movements like the elevation of the tongue tip during the 'l' sound or the deformation of the tongue around a bolus during swallowing.
- Comparison of Conditions: Figure 4(a) visually demonstrates the effect of gravity on tongue shape during speech by overlaying upright and supine surfaces, highlighting a backward rotation and tip elevation in the supine position.
Limitations and Future Work
- Limitations:
- The paper focuses on 2D sagittal ultrasound data. While this provides valuable information, it doesn't capture the full 3D dynamics of the tongue.
- The kriging model assumes a certain statistical structure for the tongue surface and noise, which might not perfectly hold in all scenarios.
- The "SURFACES" software is presented as a tool for visualization and averaging; further development for advanced statistical analysis might be needed.
- Future Work:
- The authors suggest that this methodology will further help in the quantification and statistical comparison of complex tongue motion.
- Extending the approach to 3D ultrasound data would provide a more complete understanding of tongue dynamics.
- Further exploration of the "SURFACES" software for more sophisticated statistical analyses (e.g., detailed inter-subject variability studies) is implied.
Summary updated