APPLYING IMPROVED SPECTRAL MODELING FOR HIGH QUALITY VOICE CONVERSION

Fernando Villavicencio, Axel Röebel2015

TL;DR

This paper tackles the challenge of achieving high-quality voice conversion (VC) by proposing improved spectral modeling techniques. They introduce a novel "True-Envelope" based approach that allows for more accurate spectral envelope estimation and adaptive model order selection, leading to more natural-sounding converted speech compared to traditional Linear Prediction methods.

What Problem Does This Paper Address?

Voice conversion aims to transform a speaker's voice to sound like another speaker while preserving the linguistic content. A major hurdle in achieving high-quality VC has been the accurate modeling and conversion of the spectral envelope, which defines the timbre of a voice. Existing methods, particularly those relying on Linear Prediction (LP), often suffer from oversmoothing and artifacts, leading to unnatural-sounding speech. Furthermore, the fixed dimensionality of features used in many VC systems limits their ability to adapt to speaker-specific characteristics. This paper addresses these limitations by proposing more precise spectral modeling and a framework for adapting these models to individual speakers.

Methods

The core of this paper lies in their improved spectral modeling techniques. They leverage "True-Envelope" (TE) based estimators, which are designed to more accurately capture the prominent peaks of the speech spectrum. A key innovation is the introduction of an optimal order selection mechanism for these estimators. This order selection is dynamically determined based on the fundamental frequency (F0) of the speech signal, allowing the model to adapt its complexity to the characteristics of the speaker and the speech itself. This contrasts with traditional methods that often use a fixed, arbitrary model order.

They also propose an improved all-pole modeling technique called True-Envelope LPC (TELPC). TELPC uses the spectral envelope estimations from TE as a target for autocorrelation matching, aiming to combine the benefits of both approaches.

For the voice conversion process itself, they build upon a Gaussian Mixture Model (GMM) based framework. The proposed spectral modeling improvements are integrated into this framework, allowing for more accurate feature extraction and conversion.

A significant contribution is also their perceptually-based evaluation framework. Instead of relying solely on spectral distortion measures, they incorporate perceptual cues, including non-linear frequency scaling, middle-ear filter modeling, and loudness perception, to better assess the quality of the converted speech.

Datasets

The authors created a higher quality speech database with a sampling rate of 24kHz, which is higher than the low-medium rates typically used in previous VC research. This database consists of two male and two female speakers, with a total of two hundred short, phonetically balanced phrases recorded for each speaker. This dataset was used for both training and evaluation, with approximately 32,000-40,000 voiced feature vectors for training and 5,000 for evaluation.

Key Insights and Ideas

  • Adaptive Model Order is Crucial: The paper strongly suggests that adapting the spectral model order based on speaker characteristics (specifically F0) significantly improves conversion quality, outperforming fixed-order models.
  • True-Envelope is a Better Fit: The "True-Envelope" approach, by focusing on prominent spectral peaks, provides a more accurate representation of the spectral envelope compared to traditional LP, especially for high-quality speech.
  • Perceptual Evaluation Matters: Relying solely on objective spectral distortion measures can be misleading. A perceptually-informed evaluation framework is essential for truly assessing the quality of converted speech.
  • TELPC Offers a Good Balance: The TELPC method appears to offer a good compromise, achieving performance close to the optimal TE estimator while allowing for fixed order selection without significantly increasing feature dimensionality.
  • Higher Sampling Rates Enable Higher Quality: The use of a higher sampling rate (24kHz) allows for a more detailed spectral analysis and ultimately contributes to higher quality voice conversion.

Key Results

The experimental results, visualized in Figures 1-3, demonstrate that the proposed TE and TELPC methods, especially when combined with the order adaptation strategy, consistently yield lower perceptual errors (measured in sones) compared to standard Linear Prediction (LPC) across various gender conversion scenarios (male-to-male, female-to-female, and female-to-male).

  • In female-to-male conversion, the order adaptation showed particularly significant benefits, outperforming fixed-order cases.
  • For intra-gender conversions (male-to-male and female-to-female), the order adaptation performed comparably to the best fixed-order cases.
  • The paper notes that LPC's performance degrades significantly in female cases, likely due to aliasing issues.
  • Qualitatively, the synthesized speech using the order adaptation was perceived as more natural and cleaner than that obtained through linear prediction in informal perceptual tests.

Limitations and Future Work

  • Intra-speaker Error Generalization: The authors acknowledge that the intra-speaker error measure, while helpful, cannot be generalized without evaluating it across a larger number of speakers.
  • GMM Dimensionality: While they found the overfitting effect to be low when increasing order up to selection values, they note that increased dimensionality of feature vectors due to higher sampling rates and adaptive order selection necessitates careful evaluation of data size and GMM components.
  • Future Work: The paper implicitly suggests further exploration of the perceptual evaluation framework and its application to other VC tasks. The success with higher sampling rates also points towards continued research in high-fidelity speech processing for VC.

Summary updated

Abstract

In this work, accurate spectral envelope estimation is applied to Voice Conversion in order to achieve High-Quality timbre conver-sion. True-Envelope based estimators allow model order selection leading to an adaptation of the spectral features to the characteris-tics of the speaker. Optimal residual signals can also be computed following a local adaptation of the model order in terms of the F0. A new perceptual criteria is proposed to measure the impact of the spectral conversion error. The proposed envelope models show improved spectral conversion performance as well as increased converted-speech quality when compared to Linear Prediction. Index Terms Speech synthesis, speech analysis, cepstral analysis, spectral analysis, linear predictive coding.
BibTeX
@misc{villavicencio2015applying,
  author = {Fernando Villavicencio and Axel Röebel},
  title = {{APPLYING IMPROVED SPECTRAL MODELING FOR HIGH QUALITY VOICE CONVERSION}},
  year = {2015},
}
Figure