TL;DR
This paper introduces a novel histogram equalization (HE) technique to improve the spectral quality of HMM-based speech synthesis. By matching the cumulative distribution functions (CDFs) of spectral parameters between training data and synthesized speech, the method effectively reduces over-smoothing and enhances spectral peaks, leading to more natural-sounding speech.
What Problem Does This Paper Address?
HMM-based speech synthesis, while offering flexibility in voice adaptation, often suffers from "muffled" or over-smoothed spectral characteristics. This is because HMMs, by their nature, tend to average out spectral details, leading to a loss of clarity and naturalness compared to concatenative synthesis. Existing methods to combat this, like postfiltering or global variance (GV) modeling, either require manual tuning for each speaker or have limitations in recursive application and effectiveness with certain spectral representations like mel-LSP. The authors aim to provide an automatic, effective spectral emphasis method specifically for mel-LSP parameters in HMM synthesis.
Methods
The core of the proposed method is histogram equalization (HE), a technique commonly used in image processing to enhance contrast. The authors adapt this to spectral parameters in speech synthesis.
Here's a breakdown of their approach:
- Training Phase:
- CDF Calculation: They first compute the cumulative distribution function (CDF) for each dimension of the mel-LSP parameters extracted from the training speech data.
- CDF of Generated Parameters: Simultaneously, they generate spectral parameters from context-dependent HMMs (using training sentence contexts) and compute their CDFs.
- Emphasis Rule Training: An "emphasis rule" is then trained by establishing a relationship between the CDFs of the training data and the generated parameters. This rule essentially maps generated spectral values to desired values that would achieve the target CDF. This is done by creating a look-up table.
- Synthesis (Emphasis) Phase:
- Parameter Generation: Standard HMM-based synthesis generates a sequence of spectral parameters.
- Emphasis Application: The trained emphasis rule (look-up table) is applied to each generated spectral parameter dimension independently. This process aims to transform the generated parameters so their distribution matches that of the original training data.
- Rearrangement: To mitigate potential issues like disorder and over-emphasis caused by independent dimension processing, the emphasized mel-LSP coefficients are rearranged based on frequency delta information.
Datasets
- Training Data: A speech corpus of 2237 utterances from a female voice was used for training the HMMs and calculating the reference spectral parameter CDFs.
- Test Data: Synthesized speech samples were generated using utterances not included in the training database.
- Evaluation: A crowd-sourcing system was used for subjective evaluation, with 18 listeners rating the speech quality.
Key Insights and Ideas
- Histogram Equalization for Speech: Applying HE, a well-established image processing technique, to spectral parameters in speech synthesis is a novel approach. It leverages the statistical distribution of spectral features to guide the emphasis process.
- Automatic Parameter Tuning: The HE-based approach inherently learns the emphasis rule from the data, eliminating the need for manual parameter tuning for different speakers, which is a significant advantage over traditional postfiltering methods.
- Matching Spectral Distributions: The core idea of matching CDFs between training and synthesized data is powerful. It ensures that the emphasized spectra not only have sharper peaks but also possess a statistical distribution closer to natural speech.
- Mel-LSP Specific Challenges: The paper acknowledges that mel-LSP parameters can be prone to disorder and over-emphasis when processed independently, necessitating a post-processing rearrangement step.
Key Results
- Subjective Evaluation: The proposed "PROPOSED" method significantly outperformed the "BASELINE" (non-emphasized) speech and the "GV" (Global Variance) method in subjective speech quality tests.
- Comparison to Conventional: The proposed method achieved speech quality comparable to the "CONVENTIONAL" (postfilter-based) method.
- Advantage over Conventional: Crucially, the proposed method offers a significant advantage in parameter adjustment, being automatic, whereas the "CONVENTIONAL" method requires difficult manual tuning.
- Spectral Envelope Enhancement: Figure 4 visually demonstrates that the emphasized spectral envelope includes larger peaks and valleys compared to the non-emphasized spectrum, indicating successful spectral sharpening.
Limitations and Future Work
- Limitations:
- The paper notes that processing each dimension independently can sometimes lead to disorder and over-emphasis, which is addressed by a rearrangement step.
- While comparable to the conventional postfilter, the paper doesn't explicitly state if it surpasses it in all aspects of quality, only that it's "almost the same."
- Future Work: Not explicitly detailed in the provided text, but the success of this HE-based approach suggests potential for further exploration in applying similar statistical distribution matching techniques to other speech synthesis parameters or even different speech processing tasks. The rearrangement step also hints at potential for more sophisticated methods to handle inter-dimensional dependencies.
Summary updated