Abstract
Acoustic-to-articulatory inversion mapping aims to predict the physical movements of speech articulators from acoustic signals. This task is complicated by the "one-to-many" nature of the mapping and the need for temporal consistency in predicted trajectories. This paper proposes a method using Gaussian Mixture Models (GMMs) that incorporates articulatory dynamic features and Maximum Likelihood Estimation (MLE). By considering temporal context and movement constraints, the proposed approach successfully smooths the discontinuous trajectories typical of frame-by-frame statistical models, resulting in more accurate and physically realistic articulatory estimations.
Introduction
Acoustic-to-articulatory inversion mapping is the process of estimating the vocal tract's physical configuration based on the sound produced. This is a critical component in various speech technologies, including speech coding, recognition, and synthesis. However, it remains a notoriously difficult "one-to-many" problem, where different physical mouth positions can sometimes produce nearly identical acoustic outputs.
Traditional corpus-based methods often struggle with temporal consistency. When a model predicts the position of articulators (like the tongue or lips) frame-by-frame—typically every 10ms—the resulting trajectory often contains unrealistic jumps and discontinuities. This paper addresses these issues by moving beyond simple static mapping and introducing constraints that account for the physical dynamics of articulatory movement.
Methodology
The core of the proposed method is a Gaussian Mixture Model (GMM) used to represent the joint probability of acoustic features and articulatory positions.
Feature Extraction and Contextual Input
The researchers utilized mel-cepstral coefficients extracted via the STRAIGHT analysis method for acoustic features. Articulatory positions were obtained from Electromagnetic Articulograph (EMA) data. To provide necessary temporal context, the model uses a window of multiple acoustic frames as input rather than a single moment in time.
MLE with Dynamic Features
The primary contribution of this work is the application of Maximum Likelihood Estimation (MLE) incorporating dynamic features (velocity/delta features). Instead of simply selecting the most likely position for an individual frame, the algorithm identifies a sequence of positions that is both likely given the acoustic input and physically plausible. This approach forces the model to respect the learned statistics of movement speed and smoothness, effectively acting as a "smart" statistical smoother.
Experimental Setup
The study utilized the MOCHA (Multi-CHannel Articulatory) database, which provides parallel acoustic and articulatory data for two speakers: one female (fsew0) and one male (msak0). The dataset consists of 460 British TIMIT sentences per speaker. Articulatory data were represented as 14-dimensional vectors capturing the x- and y-coordinates of seven key articulators: the top and bottom lips, bottom incisor, tongue tip, tongue body, tongue dorsum, and velum.
Results and Discussion
The researchers compared the MLE approach against a baseline GMM and a GMM whose output was smoothed using a standard low-pass filter.
Quantitative Performance
As shown in Figure 6, increasing the number of mixtures in a standard GMM initially reduces estimation error but eventually leads to performance degradation due to increased discontinuities. The MLE approach consistently outperformed both the static GMM and the low-pass filtered GMM across different mixture counts.
For the female speaker, the Root Mean Square (RMS) error was reduced from 1.61 mm (static) to 1.45 mm (MLE), with the correlation coefficient improving from 0.73 to 0.79. Similarly, for the male speaker, the RMS error dropped from 1.53 mm to 1.36 mm, while the correlation improved from 0.74 to 0.80.
Qualitative Trajectory Analysis
Visual analysis of the predicted trajectories highlights the effectiveness of the MLE method. While standard GMMs often produce jagged, unrealistic lines—especially as the number of mixtures increases—the MLE method produces smooth curves that closely track the measured ground truth.
Figure 5 illustrates this for the Y-coordinate of the tongue tip. The 1-mixture GMM is too smooth and misses detail, while the 32-mixture GMM introduces significant jitter. The MLE approach (shown here with 64 mixtures) provides a trajectory that is both detailed and physically consistent with the measured data.
Conclusion
This research demonstrates that incorporating dynamic features and MLE into a GMM-based inversion mapping framework significantly improves both the accuracy and realism of estimated articulatory movements. By leveraging the statistical properties of speech dynamics, the model overcomes the limitations of frame-by-frame mapping. Future work may involve exploring speaker-independent models and incorporating phonetic constraints to further resolve mapping ambiguities.
Summary updated