Convolutional Neural Network-Based Age Estimation Using B-Mode Ultrasound Tongue Image

Kele Xu, Tamás Gábor Csapó, Ming FengarXiv 2021 · arXiv

Introduction

Speech audio is well-known to carry "biophysical signatures" that reveal a speaker's age, health status, and identity. However, most automated systems for speaker profiling rely exclusively on acoustic signals. This research shifts the focus to the physical structures producing those sounds—specifically the tongue—which undergoes significant morphological changes during development.

Ultrasound tongue imaging (UTI) provides a non-invasive window into these articulatory patterns. Despite its potential, UTI is notoriously difficult for humans to interpret due to low Signal-to-Noise Ratios (SNR) and "speckle noise" that obscures fine anatomical structures. Even expert clinical linguists struggle to accurately estimate a child's age from a raw tongue scan. This study investigates whether deep learning can identify subtle biophysical markers in these images that are invisible to the human eye.

Methodology

The researchers treated age estimation as a supervised regression task, employing a custom Convolutional Neural Network (CNN) architecture. The model was designed to be lightweight yet effective, featuring four 2D convolutional layers for feature extraction, max-pooling layers for dimensionality reduction, and a final dense layer of 512 units leading to a single age-prediction output.

To optimize performance on relatively small medical datasets, the team utilized the RMSprop optimizer and ReLU activation functions, with a dropout rate of 0.5 to mitigate overfitting. Data augmentation played a crucial role; the team found that random rotation (up to 5 degrees) was the most effective strategy, as it simulates the natural variations in probe placement relative to the speaker's head.

Explainability and Feature Selection

To understand the model's decision-making process, the researchers generated heatmaps to visualize the regions of interest. Interestingly, the CNN did not merely track the surface contour of the tongue. Instead, it focused significantly on deeper tissues, including the tongue root and tendon regions, suggesting that internal structural development is a key indicator of age.

Heatmap visualizations showing the regions of the tongue image prioritized by the CNN for age estimation.

Datasets

The study utilized the UltraSuite repository, focusing on two distinct cohorts:

  1. UXTD (Typically Developed): Data from 58 children (ages 5–13) with typical speech development, comprising over 54,000 frames.
  2. UPX (Speech Sound Disorders): Data from 20 children (ages 6–13) diagnosed with speech sound disorders (SSD), comprising roughly 24,000 frames.

Results and Analysis

The performance was evaluated using Mean Squared Error (MSE) and Mean Absolute Error (MAE). The results highlighted a significant disparity between typically developing children and those with speech disorders:

  • Typically Developing Children (UXTD): The model achieved a Validation MSE of 2.03, vastly outperforming the baseline (guessing the mean age), which had an MSE of 3.64.
  • Children with SSD (UPX): Estimation proved significantly more difficult, with a Validation MSE of 4.87.
Mean Squared Error (MSE) trends during training on the UPX (Speech Sound Disorders) dataset.

The increased error in the SSD cohort suggests that irregular articulatory patterns—common in children with speech disorders—disrupt the standard age-related visual markers that the network relies on. Furthermore, the researchers noted that younger subjects generally produced clearer ultrasound images, likely due to the evolving density and structure of the vocal tract.

Conclusion

This study demonstrates that biophysical parameters like age are "baked into" the visual dynamics of the tongue. By capturing patterns in deeper tissues that humans cannot reliably detect, CNNs offer a promising new tool for clinical linguistics. Future work will focus on expanding the age range to include adults and exploring whether this technology can assist in the automated diagnosis of speech sound disorders by identifying deviations from typical developmental trajectories.

Summary updated

Abstract

Ultrasound tongue imaging is widely used for speech production research, and it has attracted increasing attention as its potential applications seem to be evident in many different fields, such as the visual biofeedback tool for second language acquisition and silent speech interface. Unlike previous studies, here we explore the feasibility of age estimation using the ultrasound tongue image of the speakers. Motivated by the success of deep learning, this paper leverages deep learning on this task. We train a deep convolutional neural network model on the UltraSuite dataset. The deep model achieves mean absolute error (MAE) of 2.03 for the data from typically developing children, while MAE is 4.87 for the data from the children with speech sound disorders, which suggest that age estimation using ultrasound is more challenging for the children with speech sound disorder. The developed method can be used a tool to evaluate the performance of speech therapy sessions. It is also worthwhile to notice that, although we leverage the ultrasound tongue imaging for our study, the proposed methods may also be extended to other imaging modalities (e.g. MRI) to assist the studies on speech production.
BibTeX
@misc{xu2021convolutional,
  author = {Kele Xu and Tamás Gábor Csapó and Ming Feng},
  title = {{Convolutional Neural Network-Based Age Estimation Using B-Mode Ultrasound Tongue Image}},
  year = {2021},
  eprint = {2101.11245},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.11245},
}
Figure