TL;DR
This paper introduces a novel Bayesian approach to estimate the shape of the vocal tract during nasal sounds. By directly modeling the log-spectrum of speech signals, it bypasses the need for intermediate pole-zero modeling, offering a more robust and less variable estimation of vocal tract area functions.
What Problem Does This Paper Address?
Modeling the vocal tract (VT) is crucial for understanding speech production and for applications like speech synthesis and coding. While simple "one-tube" models work well for vowels, nasal sounds (like "m" or "n") and nasalized vowels require more complex models because the nasal cavity adds extra resonances. Traditionally, estimating the VT shape for these sounds involves first finding a pole-zero model of the speech signal and then mapping these to an acoustic model, often a branched-tube model. However, this two-step process has limitations: it's not a direct mapping, and the intermediate pole-zero estimation can be prone to errors. This paper aims to directly estimate the VT area function from the speech signal's log-spectrum, avoiding the intermediate pole-zero step and leveraging a probabilistic approach to handle the inherent complexities and constraints of VT modeling.
Methods
The core of this paper is a variational Bayesian scheme designed to estimate the vocal tract area functions. Instead of explicitly estimating a pole-zero model, the method directly models the log-spectrum of the speech signal.
Here's a breakdown of the key techniques:
- Generative Model: The log-spectrum ($y$) is modeled as a non-linear function ($f(\theta, \omega)$) of the vocal tract parameters ($\theta$) plus some noise ($\epsilon$). The parameters $\theta$ represent the reflection coefficients (RCs) of the branched tubes (pharyngeal, nasal, oral) and a nasal-oral coupling parameter.
- Branched-Tube Model: A three-part tube model is used: pharyngeal, nasal, and oral tracts. The RCs from this model are related to the cross-sectional areas of the vocal tract segments.
- Bayesian Framework: The method employs a Bayesian approach, incorporating probabilistic priors on the vocal tract parameters. Specifically, it uses Gaussian priors that encourage smoothness in the vocal tract shape. This regularization helps constrain the solution space and avoid unrealistic shapes.
- Variational Bayes with Laplace Approximation and Unscented Transform: Due to the non-linear relationship between parameters and the log-spectrum, a closed-form solution is not possible. The paper uses a variational Bayesian approach. It approximates the posterior distribution of the parameters. Crucially, it modifies the standard Laplace approximation by using the unscented transform for integration. This is a more sophisticated way to handle the non-linearities and integration compared to purely analytical methods.
- Parameter Estimation: The algorithm iteratively updates the estimates for the vocal tract parameters and the noise covariance parameters until convergence.
- Direct RC Estimation: The method directly estimates the reflection coefficients (RCs) which are then used to derive the area functions. This avoids the intermediate step of estimating pole-zero models.
What's new here is the direct estimation of VT parameters from the log-spectrum using a variational Bayesian framework, specifically incorporating the unscented transform for improved handling of non-linearities. The use of smoothness priors is also a key aspect for regularization.
Datasets
The evaluation was performed on recorded speech data consisting of 6 utterances of the /m/ sound in 5 different vowel contexts. These were spoken by 3 untrained male speakers who repeated a pseudoword ('ramadama') three times each. The data was embedded in German sentences to control for prosody. Central segments of 40 ms duration were extracted from each token. A total of 4 tokens were discarded due to being shorter than 40 ms. Speech samples were downsampled to 8 kHz, and a Hanning window was applied.
The paper does not introduce a new benchmark dataset but uses existing recordings for evaluation.
Key Insights and Ideas
- Direct Estimation is Viable: The paper demonstrates that it's possible to directly estimate vocal tract area functions from the log-spectrum without relying on intermediate pole-zero modeling. This simplifies the process and potentially improves accuracy.
- Bayesian Priors for Smoothness are Key: The use of Gaussian priors that enforce smoothness on the vocal tract shape is crucial. It helps regularize the estimation and leads to more plausible and less variable results, especially across different tokens from the same speaker.
- Unscented Transform Improves Non-linear Handling: Employing the unscented transform within the variational Bayesian framework is a sophisticated way to deal with the complex, non-linear relationships in the model, leading to more robust parameter estimation.
- Less Intra-Subject Variability: The Bayesian scheme, particularly with mild smoothness assumptions, results in significantly less variability in the estimated vocal tract parameters (RCs) for a given speaker across different tokens compared to an unregularized solver. This is a significant advantage for speaker-specific modeling.
- Model Limitations: Even with a branched-tube model, certain spectral dips (zeros) cannot be perfectly modeled. This is attributed to the limited complexity of the model, specifically the exclusion of paranasal cavities (sinuses).
Key Results
- Good Spectral Fits: The proposed Bayesian method generally achieves good spectral fits to the recorded nasal speech signals.
- Optimal Prior Variance: The study found that a prior variance of $\sigma^2 = 0.1$ yielded the smallest overall error (RMS) and the highest marginal likelihood, suggesting a good balance between fitting the data and regularization.
- Comparable Modeling Error to GN: The unregularized Gauss-Newton (GN) scheme achieved comparable modeling errors (RMS) to the Bayesian approach.
- Reduced RC Variability: A key qualitative result is that the Bayesian estimation shows much less variation in the estimated RCs for a given speaker across different tokens compared to the unregularized GN scheme (illustrated in Fig. 1).
- Spectral Estimates Similar: Despite differences in parameter estimates, the resulting spectral estimates from the Bayesian and GN methods are often very similar (Fig. 2).
- Area Function Insights: The estimated area functions (Fig. 3) show expected patterns, with the nasal estimates exhibiting less variance and varying more strongly towards the velum, as anticipated.
Limitations and Future Work
- Model Complexity: The authors acknowledge that the current branched-tube model is only partially adequate for modeling nasals. The exclusion of paranasal cavities (sinuses) is a significant limitation, as their inclusion would increase the number of parameters and potentially require more sophisticated prior assumptions.
- Frequency-Dependent Terminations: Future work could explore the use of frequency-dependent glottal and nasal terminations and more realistic glottis models to increase physical plausibility.
- Anatomical Priors: A promising direction for future research is to incorporate priors deduced from anatomical measurements (e.g., from imaging methods) to further constrain the estimation and improve accuracy.
- Marginal Likelihood for Model Selection: The paper suggests that the marginal likelihood is a valuable indicator for selecting the best prior settings, implying its use in future model selection processes.
- Open Questions: The paper highlights that while the method shows promise, there are still open questions regarding the optimal complexity of the VT model and the development of appropriate priors for more complex models.
Summary updated