TL;DR
This paper introduces a novel excitation model for parametric speech synthesis that significantly reduces the characteristic "buzziness" of synthesized speech, particularly for female voices. By decomposing the residual signal into deterministic (low-frequency) and stochastic (high-frequency) components, their method achieves higher perceived speech quality with minimal computational overhead, making it suitable for commercial applications.
What Problem Does This Paper Address?
Modern parametric speech synthesizers, like those based on Hidden Markov Models (HMMs), are praised for their flexibility and low computational footprint, making them ideal for embedded systems. However, a persistent drawback is the "buzziness" in the synthesized speech quality. This is largely attributed to the simplified modeling of the excitation signal, which is typically a basic pulse train for voiced segments and white noise for unvoiced segments. While previous attempts to improve excitation modeling have shown some success, particularly for male voices, they often fall short for female voices. This paper aims to bridge this gap by proposing a more sophisticated excitation model that better captures the nuances of the residual signal, leading to more natural-sounding speech across different voice types.
Methods
The core of this paper is the proposed Deterministic plus Stochastic Model (DSM) for the residual signal. The authors adapt the principles of the Harmonic plus Noise Model (HNM) to the residual signal itself.
Here's a breakdown of their approach:
- Pitch-Synchronous Residual Frames: First, they extract residual signals from speech and then isolate short, pitch-synchronous frames centered around Glottal Closure Instants (GCIs). These frames are windowed (Blackman) and are two periods long.
- Maximum Voiced Frequency (Fm): The residual signal is conceptually split into two spectral bands at a fixed frequency of 4 kHz (Fm).
- Deterministic Component (Low Frequencies): The low-frequency part of the residual is modeled deterministically. This is achieved by decomposing the residual frames onto an orthonormal basis derived from Principal Component Analysis (PCA). This PCA basis is learned from a large dataset of residual frames. The number of retained eigenvectors is chosen to explain about 80% of the total dispersion, balancing dimensionality reduction with information preservation.
- Stochastic Component (High Frequencies): The high-frequency part of the residual is modeled stochastically. This component is essentially a white Gaussian noise modulated by a parametric envelope and convolved with an autoregressive (AR) filter. The envelope used is a pitch-dependent triangular function, and the AR filter acts as a high-pass filter beyond Fm.
- Integration into HMM Synthesizer: This DSM excitation model is integrated into an HMM-based speech synthesizer (using the HTS toolkit). They use multi-space probability distributions (MSD) to model pitch and PCA weights, along with MGC coefficients, and their derivatives. For practical implementation, they found that using only the first eigenvector from the PCA decomposition provided significant improvements without a noticeable drop in quality, simplifying the model considerably.
Datasets
- Training Data: The paper mentions using speech data from five speakers for training: AWB (Scottish male), Bruno (French male), Julie (French female), Lucy (US female), and SLT (US female). For AWB and SLT, they used approximately 45 minutes of speech from the CMU ARCTIC database. For the other voices, they used corpora of around 2 hours provided by Acapela Group.
- Evaluation Data: A subjective test was conducted using 20 randomly chosen sentences (approximately 7 seconds each) for each of the five speakers.
Key Insights and Ideas
- The "Buzziness" Source: The paper strongly suggests that the buzziness in parametric speech synthesis is primarily an issue of excitation modeling, not just spectral envelope modeling.
- DSM for Residuals is Effective: Adapting the DSM concept, which has been successful for modeling speech itself (like in HNM), to model the residual signal is a key innovation. This allows for a more nuanced representation of the excitation.
- PCA for Deterministic Excitation: Using PCA to find an optimal orthonormal basis for decomposing the low-frequency deterministic part of the residual is an elegant way to capture its essential structure efficiently.
- Female Voices Benefit Most: A significant finding is that this DSM approach yields a much more pronounced improvement for female voices compared to previous methods, addressing a known limitation in the field.
- Simplicity Achieved: The realization that using just the first eigenvector from PCA for the deterministic component is sufficient for high-quality synthesis is a crucial insight for practical implementation, drastically reducing computational load.
Key Results
- Subjective Improvement: A Comparative Mean Opinion Score (CMOS) test with 40 participants showed a clear improvement of the proposed DSM excitation over the traditional pulse excitation for all five speakers.
- Quantified Preference: For the traditional pulse excitation, the DSM was preferred by a significant margin. For speaker SLT, preference jumped from 30% with a previous codebook method to over 90% with the DSM.
- Average CMOS Scores: Average CMOS scores ranged between 1 and 1.75, indicating a substantial advantage for the proposed technique.
- Minimal Computational Load: The simplified DSM model (using the first eigenvector) requires very little computational load and memory, making it suitable for commercial applications and small devices.
Limitations and Future Work
- Fixed Fm: The paper uses a fixed maximum voiced frequency (Fm) of 4 kHz. While justified by previous work and the nature of the residual spectrum, exploring adaptive Fm estimation could potentially offer further improvements.
- Stochastic Component Modeling: While the stochastic component is based on established HNM principles, further research into optimizing the envelope and AR model could be explored.
- Limited Speaker Set: While five speakers were used, a broader range of voices and languages could be tested to confirm the generalizability of the findings.
- Future Work: The authors highlight that the proposed model's efficiency and effectiveness make it well-suited for integration into commercial applications. Further work could focus on refining the stochastic component and exploring adaptive frequency band splitting.
Summary updated