Abstract
This paper proposes a WaveNet-based neural excitation model (ExcitNet) for statistical parametric speech synthesis systems. Conventional WaveNet-based neural vocoding systems significantly improve the perceptual quality of synthesized speech by statistically generating a time sequence of speech waveforms through an auto-regressive framework. However, they often suffer from noisy outputs because of the difficulties in capturing the complicated time-varying nature of speech signals. To improve modeling efficiency, the proposed ExcitNet vocoder employs an adaptive inverse filter to decouple spectral components from the speech signal. The residual component, i.e. excitation signal, is then trained and generated within the WaveNet framework. In this way, the quality of the synthesized speech signal can be further improved since the spectral component is well represented by a deep learning framework and, moreover, the residual component is efficiently generated by the WaveNet framework. Experimental results show that the proposed ExcitNet vocoder, trained both speaker-dependently and speaker-independently, outperforms traditional linear prediction vocoders and similarly configured conventional WaveNet vocoders.
Introduction
In Statistical Parametric Speech Synthesis (SPSS), WaveNet has emerged as a transformative technology, producing speech that sounds significantly more natural than traditional methods. However, WaveNet models often struggle to capture high-frequency details and dynamic changes because they attempt to model the entire, complex speech waveform directly. This often results in audible artifacts, such as "hissing" or noise in the output.
Previous solutions utilized noise-shaping filters to mask these errors, but these filters are typically time-invariant, failing to adapt to specific phonetic contexts like unvoiced sounds or rapid transitions. ExcitNet addresses this by combining the statistical power of WaveNet with the physical principles of the source-filter theory of speech production.
Methodology
The fundamental innovation of ExcitNet is the decoupling of the spectral structure (the "filter") from the excitation signal (the "source"). This process involves several key steps:
- Analysis: A Linear Prediction (LP) filter is used to remove the spectral formant structure from real speech, leaving a residual excitation signal.
- Modeling: WaveNet is trained specifically to predict this residual excitation signal rather than the raw audio. Because the excitation signal is less complex, the model operates more efficiently.
- Conditioning: To improve pitch and rhythm accuracy, the model is conditioned on "Improved Time-Frequency Trajectory Excitation" (ITFTE) parameters, which distinguish between periodic (Slowly Evolving Waveform) and noise-like (Rapidly Evolving Waveform) components.
- Synthesis: During inference, the model generates a synthetic excitation signal, which is then passed through an LP synthesis filter to reconstruct the final speech waveform.
Experimental Setup
The researchers evaluated ExcitNet using high-quality speech corpora from professional Korean speakers (KRF - Female, KRM - Male). To test generalization, they also developed a Speaker-Independent (SI) model using a 10-hour dataset featuring 10 different speakers.
Results and Discussion
ExcitNet demonstrated superior performance across both objective and subjective metrics. Objective tests showed that ExcitNet achieved the lowest Log-Spectral Distance (LSD) and F0 (pitch) Root Mean Square Error compared to standard WaveNet and noise-shaping WaveNet (WN-NS) baselines.
In subjective Mean Opinion Score (MOS) tests, the Speaker-Dependent (SD) ExcitNet achieved scores of 4.35 for the female speaker and 4.47 for the male speaker, approaching the quality of natural raw recordings (~4.6). Listeners also showed a strong preference for ExcitNet in head-to-head tests, choosing it over WN-NS 83.6% of the time for female speech.
Key insights from the study include:
- Efficiency through Decoupling: By allowing a traditional filter to handle the speech "shape," WaveNet can focus entirely on the excitation energy, leading to cleaner output.
- Adaptive Transitions: Unlike static noise-shaping methods, ExcitNet's adaptive approach maintains high quality during difficult unvoiced-to-voiced transitions.
- ITFTE Impact: The inclusion of ITFTE parameters significantly reduced training error (Negative Log-Likelihood), providing the network with essential information regarding voice periodicity.
Conclusion
ExcitNet represents a significant advancement in neural vocoding by successfully merging deep learning with classical speech production theory. While the model currently operates within a multi-stage pipeline and faces some challenges with high-pitched voices in speaker-independent settings, it provides a robust foundation for future end-to-end speech synthesis systems.
Summary updated