Voice conversion using k-histograms and frame selection

Alejandro José Uriz, Pablo Daniel Agüero, Antonio Bonafonte, Juan Carlos TulliInterspeech 2009

TL;DR

This paper introduces a novel voice conversion approach using k-histograms for non-Gaussian statistical transformation of speech features, aiming to improve target speaker similarity and converted speech quality. They also propose a second method that combines this k-histogram transformation with frame selection, demonstrating competitive results against established methods.

What Problem Does This Paper Address?

Voice conversion (VC) aims to modify a source speaker's voice to sound like a target speaker. While statistical methods like Gaussian Mixture Models (GMMs) have advanced VC, they often introduce over-smoothing, degrading speech quality. Existing techniques face a trade-off between preserving target speaker identity and maintaining natural speech quality. This paper seeks to overcome this by proposing a non-Gaussian transformation method that avoids assumptions about parameter distributions and by integrating frame selection to better preserve target speaker characteristics.

Methods

The core innovation lies in using k-histograms for clustering speech features, specifically Line Spectral Frequency (LSF) parameters. Unlike k-means, k-histograms represent clusters by their feature distributions (histograms), avoiding assumptions about the underlying data distribution.

  1. K-Histograms Transformation: LSF parameters are discretized and clustered using k-histograms. The conversion from source to target LSF parameters is then performed coefficient-by-coefficient using the Cumulative Density Function (CDF) of the respective clusters. This is a non-Gaussian to non-Gaussian mapping.
  2. Frame Selection Stage: For the second proposed method (FSKH), after the k-histogram transformation, a frame selection stage is added. This stage uses a Viterbi algorithm to find an optimal sequence of target speaker frames from the training data that best matches the converted source features. This aims to avoid the smoothing introduced by direct statistical mapping. The cost function for frame selection considers LSF, energy, fundamental frequency, and phone identity with zone information. To manage computational load, only target frames belonging to the cluster assigned to the converted source frame are considered.

The paper compares these methods against GMM-based VC and Dutoit's frame selection method.

Datasets

The audio database consisted of 200 sentences in Spanish, spoken by two male and two female speakers. The sampling frequency was 16 KHz.

  • 50% of sentences were used for training conversion functions.
  • 30% were used as a development set for tuning model parameters.
  • 20% were used for objective and subjective testing (38 unseen sentences for conversion and resynthesis).

Four conversion directions were tested: male-to-male (m2m), female-to-female (f2f), male-to-female (m2f), and female-to-male (f2m).

Key Insights and Ideas

  • Non-Gaussian Assumption is Key: The paper argues that avoiding Gaussian assumptions in feature transformation (via k-histograms) can lead to better voice conversion results than traditional GMMs.
  • K-Histograms as a Viable Alternative: K-histograms prove to be an effective clustering method for LSF parameters, offering a good balance between similarity and quality.
  • Frame Selection Synergizes: Combining k-histogram transformation with frame selection (FSKH) shows promise, though it can sometimes degrade quality compared to direct transformation alone.
  • Quantization Impact: The degree of LSF parameter quantization for histogram estimation did not significantly impact the subjective MOS scores, suggesting robustness.
  • FSOPT as a Benchmark: The FSOPT method (finding the closest target frame without transformation) serves as a high-quality benchmark, indicating the potential achievable with frame selection.

Key Results

  • Subjective Similarity (MOS-S): Methods based on k-histograms (KH3140, KH314) achieved a MOS-S of 3.6, matching the best frame selection method (FSOPT) and outperforming GMM (2.7) and DUTOIT (3.4).
  • Subjective Quality (MOS-Q): The k-histogram methods (KH3140: 3.0, KH314: 2.8) were competitive, with KH3140 matching FSOPT (2.8) and outperforming GMM (2.1).
  • FSKH Performance: The FSKH methods (with frame selection) showed slightly lower MOS-S and MOS-Q scores (3.1-3.4 and 2.3 respectively) compared to direct k-histogram transformation, suggesting frame selection might introduce some degradation in this context.
  • GMM Improvement with Frame Selection: Interestingly, for GMM, the addition of frame selection (Dutoit's proposal) improved similarity by 0.7 points and quality by 0.3 points compared to direct GMM transformation.
  • Objective Results (P-distance): Objective P-distance scores showed no statistically significant differences between the methods, leading the authors to focus on subjective evaluations.

Limitations and Future Work

  • Quantization Noise: The discretization of LSF parameters for histogram estimation introduces noise, which the authors acknowledge as a drawback.
  • Computational Load: The frame selection stage can be computationally intensive, though k-histograms helps reduce the search space.
  • Excitation Not Modeled: The current methods primarily focus on spectral envelope conversion (LSF). The fundamental frequency contour is transformed, but the source excitation (residual signal) is used directly.
  • Future Work: The authors plan to extend the system to include excitation modeling and transformation to further improve the overall voice conversion quality, making it more suitable for real-life applications. They also aim to incorporate state-of-the-art techniques for excitation handling.

Summary updated

BibTeX
@inproceedings{uriz2009voice,
  author = {Alejandro José Uriz and Pablo Daniel Agüero and Antonio Bonafonte and Juan Carlos Tulli},
  title = {{Voice conversion using k-histograms and frame selection}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure