TL;DR
This paper introduces a novel approach to HMM-based Text-to-Speech (TTS) synthesis by incorporating the Sub-band Basis Spectrum Model (SBM). SBM allows for a more accurate representation of vocal tract spectra and phase characteristics, leading to improved speech quality compared to traditional methods, particularly in capturing finer spectral details and phase information.
What Problem Does This Paper Address?
Traditional HMM-based TTS systems often struggle with generating natural-sounding speech due to limitations in how they model acoustic features and excitation signals. Specifically, commonly used features like mel-cepstrum (MCEP) can lose crucial high-frequency spectral details, resulting in muffled speech. While alternatives like mel-scaled line spectral pair (MLSP) offer sharper spectral peaks, they can lead to instability in the synthesis filter. Furthermore, simple excitation models fail to accurately capture the nuances of human vocal source signals, contributing to unnatural-sounding speech. This paper aims to address these limitations by proposing the integration of the SBM, which promises a richer representation of both spectral and phase information, into HMM-based TTS.
Methods
The core of this work is the integration of the Sub-band Basis Spectrum Model (SBM) into an HMM-based TTS framework. SBM represents vocal tract spectra and phase characteristics as a linear combination of sub-band basis vectors. These basis vectors are designed with specific frequency characteristics, including asymmetrical unimodal shapes and overlapping bands, and are formulated to align with perceptual scales (mel-scale for lower frequencies, linear for higher).
The proposed system modifies the standard SBM parameters for HMM-based TTS:
- Vocal Tract Spectrum Parameter (VTSP): To overcome issues with the original VTSP's correlation and training difficulties, a Discrete Cosine Transform (DCT) is applied. This "DCT-VTSP" is argued to retain more essential spectral factors compared to MCEP, which often reduces dimensionality by discarding high-frequency coefficients.
- Phase Spectrum Parameter (PSP): To better model phase information, the paper introduces "complex PSP" (CPSP). This involves representing the phase as a complex number (cosine + sine) and applying SBM analysis to both the real and imaginary parts. This approach simplifies phase wrapping issues and allows for a more direct integration of phase information into the HMM framework.
The overall TTS system uses context-dependent Hidden Semi-Markov Models (HSMMs) for parameter generation. Waveform generation involves decoding CPSP using sub-band basis vectors, generating multi-pulse trains based on log-F0 and decoded phase, creating a mixed excitation signal (combining multi-pulse trains and white noise based on banded aperiodic components), and finally filtering this excitation with the vocal tract spectral sequence derived from DCT-VTSP.
Datasets
The paper mentions using a "Speech DB" for training and evaluation. Specific details about the size and nature of this database are not provided beyond mentioning a Japanese male speaker and two Japanese female speakers with different numbers of utterances (853, 572, and 2000). No new benchmarks are introduced.
Key Insights and Ideas
- SBM's Potential for Richer Representation: The paper highlights SBM's ability to represent both spectral and phase information more comprehensively than traditional features like MCEP or MLSP. This is crucial for natural speech synthesis, as both aspects contribute significantly to perceived quality.
- DCT for VTSP Stability: Applying DCT to VTSP is a clever way to decorrelate the parameters and make them more amenable to HMM training, addressing a key limitation of raw VTSP.
- Complex PSP for Phase Modeling: The introduction of complex PSP offers a more robust and direct method for incorporating phase information into HMM-based TTS, circumventing issues like phase wrapping and simplifying parameter estimation.
- Conditional Improvement: The results suggest that the proposed SBM-based parameters don't universally outperform all previous systems but show significant improvements under specific conditions, particularly for female speakers and when phase information is utilized.
Key Results
Subjective evaluations were conducted using a 5-level mean opinion score.
- Male Speaker: The proposed systems (S50N and S50P) achieved better speech quality than C40N, C50N, and C50P. However, L40N (using MLSP) achieved the highest score.
- Female Speaker 1: S50N and S50P significantly outperformed other HMM-based TTS systems. Systems incorporating the phase parameter (P suffix) showed a slight improvement in voice quality.
- Female Speaker 2: S50P achieved the best speech quality. Again, the phase parameter offered a slight improvement.
Overall, the paper concludes that the proposed HMM-based TTS using SBM parameters "conditionally achieved better speech quality" and is "promising for realization of natural synthesized speech."
Limitations and Future Work
- Conditional Improvement: The authors acknowledge that the proposed method does not universally outperform all previous systems, with one male speaker system (L40N) achieving a better score. This suggests that the effectiveness of SBM might be speaker-dependent or dependent on the specific acoustic conditions.
- No Significant Differences in Some Cases: For the second female speaker, while S50P performed best, the paper notes that "there were no significant differences" in overall HMM-based TTS speech quality, implying that the gains might be subtle in some scenarios.
- Future Work: While not explicitly detailed, the findings suggest that further exploration into optimizing SBM parameters, exploring different basis vector designs, and potentially combining SBM with other advanced TTS techniques could lead to even greater improvements in speech naturalness. The conditional nature of the improvements also points to the need for more speaker-adaptive or context-aware integration of SBM.
Summary updated