TL;DR
This paper introduces a novel voice conversion (VC) approach that incorporates detailed glottal source modeling, leading to significant improvements in both voice quality and speaker identity transformation. By treating the glottal source and aspiration noise as distinct, modelable components alongside the vocal tract, the authors demonstrate a more natural and accurate synthesis of speech.
What Problem Does This Paper Address?
Voice conversion (VC) aims to transform a source speaker's voice to sound like a target speaker's voice. Traditional VC systems often rely on simplified models of speech production, particularly for the voice source (the sound generated by the vocal folds) and aspiration noise (the breathy component of speech). These simplified models can lead to unnatural-sounding synthesized speech and imperfect speaker identity transfer. This paper addresses the gap by investigating whether a more accurate, source-filter based representation of speech production, explicitly modeling the glottal source and aspiration noise, can enhance VC performance. The authors build upon the idea that speech is a product of the glottal source exciting the vocal tract, and then radiating from the lips, aiming to capture these components more faithfully.
Methods
The core of this paper is a new parametrization of speech signals that goes beyond standard vocal tract modeling. The authors employ a previously developed algorithm for source-filter decomposition to separate the speech signal into its constituent parts:
- Vocal Tract: Modeled using Line Spectral Frequencies (LSF), a standard representation derived from Linear Predictive Coding (LPC) analysis.
- Glottal Source: Modeled using the LF (Liljencrants-Fant) model, a more flexible representation of the glottal waveform than simpler models.
- Aspiration Noise: This is a novel addition. The authors extract aspiration noise from the parametrization error of the glottal source model. This noise is then modeled using an amplitude-modulated Gaussian noise, where the modulation envelope is parametrized using a few key parameters.
This new parametrization is then integrated into a decision tree-based voice conversion system. This system uses CART (Classification and Regression Trees) to classify speech frames into phonetic categories, and then employs Gaussian Mixture Models (GMMs) to learn the transformation function for each category. The authors train separate CART models for the vocal tract, glottal source, and aspiration noise parameters.
Datasets
The study utilizes a Spanish voice conversion dataset collected as part of the TC-STAR project. This dataset includes recordings from four bilingual speakers (two female, two male), with approximately 200 phonetically rich sentences per language. For this work, only the Spanish data was used. Ten sentences were reserved for testing, and the rest for training. The authors trained four different source-target pairs: m1 to m2, m1 to f2, f1 to m2, and f1 to f2.
Key Insights and Ideas
- The Glottal Source Matters: Explicitly modeling the glottal source, beyond just its spectral impact, significantly improves voice quality and speaker identity. This suggests that the nuances of vocal fold vibration are crucial for natural-sounding speech.
- Aspiration Noise is Key to Naturalness: Recovering and modeling aspiration noise from the parametrization error is vital for avoiding a "vocoder-like" unnatural sound. This highlights the importance of capturing the breathy, turbulent aspects of speech.
- Parametrization is Crucial for VC: The paper strongly advocates for the idea that the underlying signal representation has a profound impact on VC performance, even when using established VC algorithms. Better parametrization can unlock the potential of existing methods.
- Source-Filter Decomposition as a Foundation: The authors demonstrate the power of a robust source-filter decomposition technique as a foundation for advanced speech processing tasks like VC.
Key Results
The evaluation, based on subjective listening tests by 14 human judges, showed significant improvements:
- Overall Voice Quality: Improved from a Mean Opinion Score (MOS) of 2.11 (baseline) to 2.47 (proposed parametrization).
- Speaker Identity Similarity: Improved from a MOS of 2.87 (baseline) to 3.02 (proposed parametrization).
- Qualitative Improvement: The new parametrization resulted in a noticeable improvement in transformed voice quality across all tested conversion pairs. While the similarity improvement was not uniform across all pairs, it was particularly successful in transforming towards the second female voice (f2).
Limitations and Future Work
- Quality Still Lags: Despite improvements, the quality of the synthesized speech is still considered poor compared to equivalent speech synthesis systems.
- Aspiration Noise Modeling: The authors believe that further improvements in modeling aspiration noise could enhance overall synthetic quality.
- Conversion Paradigms: They plan to investigate better conversion paradigms that can leverage the proposed parametrization more effectively.
Summary updated