Iterative MMSE estimation of vocal tract length normalization factors for voice transformation

Daniel Erro, Eva Navas, Inma HernáezInterspeech 2012

TL;DR

This paper introduces a novel iterative method for estimating optimal vocal tract length normalization (VTLN) parameters for voice transformation. By minimizing an objective error in the cepstral domain, their approach achieves comparable conversion accuracy to statistical methods while significantly improving the quality of the transformed speech.

What Problem Does This Paper Address?

Voice transformation (VT) aims to alter a speaker's voice to sound like another speaker. Traditional statistical methods, like those based on Gaussian Mixture Models (GMMs), often struggle with capturing fine spectral details, leading to an "oversmoothing" effect and less natural-sounding speech. Frequency warping (FW) offers a more physically motivated approach by mapping spectral frequencies, preserving speech quality better. However, existing FW methods often rely on computationally expensive grid searches for parameter estimation and can still suffer from inaccuracies, especially in amplitude. This paper addresses the challenge of efficiently and accurately estimating VTLN parameters for FW-based voice conversion, aiming to bridge the gap between statistical methods' accuracy and FW's quality preservation.

Methods

The core contribution is an iterative MMSE (Minimum Mean Squared Error) estimation method for a single parameter of a bilinear VTLN function. This method iteratively refines the warping factor by minimizing the cepstral error between the warped source speech and the target speech.

Key aspects of their method include:

  • Iterative MMSE Estimation: Instead of grid search, they propose an iterative procedure that converges to an optimal warping factor by repeatedly calculating and applying incremental adjustments. This is derived from minimizing the MMSE in the cepstral domain.
  • Linearized Approximation: For efficiency, they utilize a linearized approximation of the warping matrix, which is valid when the warping factor is small.
  • Extension to Multiple Warping Classes: The method is extended to handle scenarios where different acoustic classes (e.g., voiced/unvoiced, or phonetically distinct segments) might require different warping factors. This is achieved by defining an instantaneous warping factor that is a weighted sum of class-specific factors, with weights determined by the probability of a frame belonging to each class (soft clustering).
  • Amplitude Scaling: To further improve conversion accuracy, they introduce a complementary amplitude scaling method. This is implemented as a bias vector added to the cepstral representation of the warped spectrum, also estimated iteratively using MMSE.

Datasets

The authors used the CMU ARCTIC database for their experiments. They selected four speakers: two female (slt, clb) and two male (bdl, rms). Fifty parallel training sentences and a separate set of fifty sentences for testing were used for each speaker. Phonetic segmentation was employed for frame-level alignment.

Key Insights and Ideas

  • Parametric FW is Viable: The paper demonstrates that a physically motivated parametric approach like VTLN, when optimized iteratively, can achieve performance comparable to statistical methods.
  • Iterative Optimization is Efficient: The proposed iterative MMSE estimation is more efficient than traditional grid search for VTLN parameters.
  • Amplitude Scaling is Crucial: The addition of amplitude scaling significantly improves conversion accuracy, especially in cases where VTLN alone is insufficient.
  • Quality Gains are Significant: The primary advantage of their method lies in the improved quality of the converted speech, even when conversion accuracy is similar to baseline methods.
  • Cross-Gender VC Challenges: The bilinear VTLN function's limitations become apparent in cross-gender conversions, where significant differences in vocal tract length pose a challenge.

Key Results

  • Subjective Evaluation: MOS (Mean Opinion Score) tests showed that the proposed method achieved comparable average converted-target similarity to a GMM-based baseline.
  • Quality Improvement: Crucially, the proposed method yielded significantly better average quality scores than the GMM baseline.
  • Intra-Gender Performance: The method consistently outperformed the GMM baseline in intra-gender conversions.
  • Cross-Gender Limitations: While still offering quality improvements, the method was less successful in cross-gender conversions due to the limitations of the bilinear warping function.

Limitations and Future Work

  • Bilinear Function Limitations: The authors acknowledge that the bilinear VTLN function's shape might restrict its warping capability, particularly in cross-gender scenarios with large vocal tract differences. This can lead to less natural transformations as the system tries to compensate for mismatches.
  • Future Work: They suggest exploring more sophisticated warping curves that can be applied to cepstral envelopes, which would necessitate redesigning the MMSE training procedure.

Summary updated

BibTeX
@inproceedings{erro2012iterative,
  author = {Daniel Erro and Eva Navas and Inma Hernáez},
  title = {{Iterative MMSE estimation of vocal tract length normalization factors for voice transformation}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure