TL;DR
This paper tackles the "muffled sound" issue in GMM-based voice conversion by introducing a discriminative training (DT) method. By refining the GMM parameters to reduce ambiguities between acoustic classes, the DT approach significantly improves the naturalness and speaker individuality of converted speech.
What Problem Does This Paper Address?
Voice conversion (VC) aims to transform a speaker's voice to sound like another. A common challenge in GMM-based VC, particularly spectral conversion, is the "muffled sound" effect. This occurs when the model over-smooths the spectral features, leading to a loss of clarity and detail. The authors attribute this over-smoothing to ambiguities among the "generative classes" (Gaussian mixture components) that model the relationship between source and target speech features. When a single set of source features can map to multiple target features, the resulting converted voice can sound unclear. Existing methods have tried to address this through techniques like dynamic frequency warping or incorporating global variance, but this paper proposes a new approach focused on improving the discriminative power of the GMM itself.
Methods
The core of this paper is a novel discriminative training (DT) method applied to a Joint Density GMM (JDGMM).
Here's a breakdown:
- JDGMM: This is the standard model used for GMM-based voice conversion. It models the joint probability distribution of source and target speech features. The training typically uses Maximum Likelihood (ML) estimation.
- The Problem with ML Training: ML training aims to maximize the probability of the observed data. However, it can lead to a situation where multiple generative classes (Gaussian components) are highly ambiguous for a given source feature. This ambiguity is quantified by the authors using the posterior probability of a generative class given the source features. A lower posterior probability indicates higher ambiguity.
- Discriminative Training (DT): The proposed DT method aims to directly optimize a criterion that reduces these ambiguities. Specifically, it maximizes the posterior probability of the correct generative class for a given source feature. This is analogous to Maximum Mutual Information (MMI) training in speech recognition but applied to voice conversion.
- Objective Function: The DT objective is to maximize a function related to the posterior probability of the generative classes, effectively pushing apart the distributions of different classes.
- Parameter Updates: The DT method refines specific parameters of the JDGMM, namely the transformation matrix (A), bias vector (b), and covariance matrices (Σ(XX)), using a gradient-based approach. Other parameters like prior probabilities (ω) and mean vectors (μ) are kept from the ML-trained model.
Datasets
The experiments were conducted on a parallel Mandarin speech corpus.
- Speakers: One female (source) and one male (target).
- Data Split: 40 sentences were used for training the conversion system, and the remaining 40 sentences were used for conversion and evaluation.
- Speech Signal Processing: Signals were recorded at 20kHz and down-sampled to 16kHz. Spectral features used were the first 24 Mel-cepstral coefficients (MCCs) extracted from the STRAIGHT smoothed spectra.
Key Insights and Ideas
- Ambiguity as the Root Cause: The paper strongly posits that the "muffled sound" effect is not just about over-smoothing, but specifically about the ambiguity between different acoustic (generative) classes in the GMM.
- DT for Clarity: Discriminative training, by forcing classes to be more distinct, directly combats this ambiguity. This is a crucial shift from purely generative modeling.
- Posterior Probability as a Key Metric: The posterior probability of a generative class serves as a powerful objective measure for both the degree of ambiguity and the effectiveness of the DT method.
- Lower Order MCCs Matter More for Naturalness: The DT method's impact on enhancing spectral trajectories is more pronounced for lower-order MCCs. This aligns with prior findings that higher-order MCCs have naturally smaller dynamic ranges, and enhancing them might not lead to better perceived quality.
Key Results
- Objective Evaluation:
- Posterior Score: The DT-trained model (ML+DT) achieved a significantly higher posterior probability score (-2.25) compared to the ML-trained model (-4.57). This indicates a substantial reduction in ambiguities among generative classes. The "oracle" case (where the correct class is known) achieved -0.11, showing the DT method gets close to ideal performance.
- Mel-cepstral Distortion (MCD): The ML+DT method resulted in a slightly higher MCD (5.85 dB) compared to the ML method (5.12 dB). While this might seem counterintuitive, the authors argue it's a trade-off for improved clarity.
- Subjective Evaluation:
- Speech Quality & Speaker Individuality: Formal listening tests showed that listeners preferred the speech converted using the ML+DT method over the ML method for both speech quality and speaker individuality.
- Alleviation of Muffled Sound: Subjects consistently reported that the DT method effectively reduced the "muffled sound" effect.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the DT method results in a slightly higher MCD, suggesting a potential trade-off between clarity and spectral accuracy.
- The DT was applied only to specific parameters (transformation, bias, covariance) for simplicity and based on preliminary experiments showing limited gains from updating all parameters.
- The evaluation was performed on a single speaker pair (female-to-male) and a specific language (Mandarin).
- Future Work:
- Investigate the connection between the "one-to-many" mapping problem and weak correlation problems with the proposed DT method.
- Conduct evaluations on more speaker pairs and potentially different languages.
- Explore applying DT to the full set of JDGMM parameters.
Summary updated