TL;DR
This paper explores how to build accurate "forward models" of the vocal tract, which map how we move our articulators (tongue, jaw, etc.) to the resulting sound. They found that Locally-Weighted Regression (LWR) is a more practical and often more accurate method than traditional Artificial Neural Networks (ANNs) for this task, especially when local linear approximations are useful for control.
What Problem Does This Paper Address?
Understanding how we produce speech involves modeling the complex relationship between the physical movements of our vocal tract articulators and the resulting acoustic output. This is known as the "forward kinematics" problem. While Artificial Neural Networks (ANNs) are powerful function approximators capable of learning such complex, nonlinear mappings, they can be difficult to design and slow to train. Furthermore, many control theories for speech production rely on local linear approximations of this forward model (like using a Jacobian matrix). ANNs don't directly provide this local linearity, requiring an extra step to derive it. This paper addresses the need for a more practical and theoretically aligned method for learning these vocal tract forward models, particularly for applications in speech control.
Methods
The authors compare two machine learning techniques for learning the vocal tract's forward kinematics:
- Artificial Neural Networks (ANNs): They implemented a standard feedforward neural network with two hidden layers and sigmoidal activation functions. Training was done using backpropagation. They also discuss how to derive a linearized representation (Jacobian) from a trained ANN, either numerically or analytically.
- Locally-Weighted Regression (LWR): This is a memory-based, "lazy learning" method. Instead of training a global model, LWR fits a local linear model to the data points closest to the prediction point, using a Gaussian kernel to define locality. A key advantage highlighted is that the locally-linear model is directly available, making it straightforward to obtain the Jacobian.
Datasets
The data used for training and evaluation was generated using the Task Dynamic Application (TADA), an articulatory speech synthesizer. TADA is built upon the Configurable Articulatory Synthesizer (CASY) model, which provides both a geometric vocal tract model and analytically derived Jacobians. The articulatory variables (e.g., jaw height, tongue body position) were systematically manipulated to create the dataset. A subset of 2000 data points was randomly selected for experiments.
Key Insights and Ideas
- LWR aligns better with control theory: Many speech control theories assume local linearity. LWR directly provides this local linear approximation (the Jacobian), whereas ANNs require an additional step to derive it.
- Simplicity and efficiency: LWR has fewer free parameters and a simpler training process compared to ANNs, making it more practical for this application.
- Task-specific difficulty: The accuracy of both methods varied significantly across different speech tasks. For instance, estimating the tongue tip constriction degree (TTCD) was particularly challenging for both ANNs and LWR, likely due to its high nonlinearity.
- Bias-variance tradeoff: Adjusting LWR's kernel width (h) and ANN's number of hidden nodes influences their bias-variance tradeoff, impacting accuracy. However, LWR generally outperformed the tested ANN configuration across a range of these parameters.
Key Results
- LWR outperforms ANN in accuracy: For the specific vocal tract forward map explored and the chosen parameters, LWR achieved higher accuracy in estimating the Jacobian compared to the ANN.
- Quantitative comparison: Tables 1 and 2 show mean correlation coefficients between analytically derived and estimated Jacobian rows. For example, for the tongue body constriction degree (TBCD), LWR achieved a correlation of 0.86 with a mean angle difference of 30 degrees, while the ANN achieved 0.55 with a 56-degree difference.
- Performance trends: Figure 2 illustrates that LWR's performance generally improved with larger kernel widths (more locality), while ANN performance improved with more hidden nodes, though LWR maintained an advantage.
Limitations and Future Work
- Task-specific limitations: Both methods struggled with certain tasks, particularly TTCD, indicating the inherent complexity of these mappings.
- Parameter tuning: While LWR was hand-tuned, the authors note that Bayesian methods exist for automatically optimizing its free parameter.
- Future directions: The authors plan to extend this work to learn acoustic tasks and to estimate forward maps from real articulatory data (e.g., from real-time MRI). They also aim to improve learning for difficult tasks using a Bayesian approach.
Summary updated