Abstract
This paper utilizes global variance (GV) of the log power spectrum (LPS) derived from mel-cepstum to improve hidden Markov model (HMM) based parametric speech synthesis. In order to alleviate the over-smoothing effect on the generated spectral structures, an LPS-GV modeling method using line spectral pairs (LSPs) has been proposed in our previous work, where the estimated distribution of LPS-GV was combined with the trained acoustic models to determine the optimal spectral features at synthesis time. In this paper, we extend this method to the condition where mel-cepstral coefficients are used as spectral features. Further, a method of integrating LPS-GV distortions into the criterion of minimum generation error (MGE) model training is proposed in order to avoid high computational complexity of the parameter generation algorithm considering GV model. Experimental results show that the parameter generation algorithm using LPS-GV model produces more natural acoustic features than the conventional GV modeling method when mel-cepstrum features are used. Besides, integrating LPS-GV distortions into model training criterion achieves similar performance as applying LPS-GV model at synthesis time.
Introduction
Hidden Markov Model (HMM) based speech synthesis is a widely used parametric method, yet it frequently suffers from an "over-smoothing" effect. Because HMMs typically predict mean spectral values, the resulting synthetic speech can sound muffled or artificial compared to natural human speech.
To address this, researchers often employ Global Variance (GV) techniques to ensure that the generated parameters exhibit a variance over time similar to that of natural speech. However, standard GV methods often treat abstract features like mel-cepstral coefficients as independent dimensions and require computationally intensive iterative optimization during the synthesis stage. This paper proposes modeling the GV of the Log Power Spectrum (LPS) and integrating this constraint directly into the training phase via Minimum Generation Error (MGE) training.
Methodology
LPS-GV on Mel-Cepstrum
The authors propose deriving the Log Power Spectrum (LPS) from mel-cepstral coefficients before applying GV constraints. Unlike raw cepstral coefficients, the LPS domain more closely aligns with human auditory perception. By modeling variance in the LPS domain, the system implicitly captures the correlations between different mel-cepstral dimensions, leading to a more realistic spectral envelope.
MGE Training Integration
To eliminate the computational burden of iterative optimization at runtime, the authors integrated the LPS-GV constraint into the Minimum Generation Error (MGE) training criterion. By adding an "LPS-GV distortion" component to the MGE error function, the model parameters are optimized during training to inherently produce the desired variance. This allows for high-quality synthesis at the speed of standard HMM-based systems.
As shown in the figure above, the reduction in LPS-GV distortion is highly sensitive to the weight parameter $w^{(t)}$ used during training. Increasing this weight significantly reduces distortion across various spectral points, though a balance must be struck to avoid increasing other feature distortions.
Experimental Results
The proposed methods were evaluated using a Chinese speech database consisting of 1,100 sentences from a professional female speaker. The system utilized 41-order mel-cepstrum and log F0 features extracted via the STRAIGHT vocoder.
Subjective evaluation was conducted using Mean Opinion Scores (MOS) to measure naturalness. The results indicated that the proposed MGE_LPS-GV approach outperformed both the baseline HMM and standard MGE training.
Key findings from the evaluation include:
- Naturalness: The MGE_LPS-GV system achieved the highest MOS (~3.5), significantly improving upon the baseline (~2.5).
- Preference: In head-to-head tests, listeners preferred the LPS-GV method over conventional GV modeling 63.6% of the time.
- Efficiency: Integrating the variance constraint into the training stage (MGE_LPS-GV) proved as effective as applying it at synthesis time, while maintaining high synthesis speed.
Conclusion
By modeling Global Variance in the Log Power Spectrum domain and incorporating it into Minimum Generation Error training, this research provides a pathway to more natural-sounding HMM-based speech synthesis without the typical computational overhead. Future work will focus on further refining the weight tuning process and comparing the effectiveness of LPS-GV across different feature sets, such as Line Spectral Pairs (LSPs).
Summary updated