Sinusoidal model parameterization for HMM-based TTS system

Slava Shechtman, Alex SorinInterspeech 2010

TL;DR

This paper introduces a novel way to represent speech for HMM-based Text-to-Speech (TTS) systems by using Mel-frequency Regularized Cepstral Coefficients (MRCC) to model the amplitude spectrum of sinusoidal speech representations. This approach leads to significantly improved synthesized speech quality compared to existing methods, particularly in capturing finer spectral details.

What Problem Does This Paper Address?

Modern HMM-based TTS systems often rely on source-filter models to synthesize speech. While effective, these models can sometimes produce less natural-sounding speech. An alternative is the sinusoidal model, which represents speech as a sum of harmonic components. However, efficiently parameterizing this sinusoidal model for HMMs has been a challenge. Existing methods, like using Mel-Generalized Cepstral-Linear Predictive Spectral Pairs (MGC-LSP), are not optimally suited for sinusoidal reconstruction, leading to potential spectral smearing. This paper aims to bridge this gap by proposing a new parameterization method that better fits the sinusoidal model, thereby enhancing the quality of HMM-based TTS.

Methods

The core of this paper is the introduction of Mel-frequency Regularized Cepstral Coefficients (MRCC) as a parameterization for the amplitude spectrum of voiced speech within a sinusoidal model.

Here's a breakdown of the key techniques:

  • Sinusoidal Model: The paper adopts a constant-frame sinusoidal model, which represents voiced speech as a sum of harmonic sinusoids. This model is known for its ability to produce high-quality speech and allows for modifications like pitch shifting. The analysis involves estimating harmonic amplitudes, frequencies, and phases.
  • Regularized Cepstral Coefficients (RCC): To represent the continuous amplitude spectrum efficiently, the authors utilize RCC. This method models the spectrum as a sum of cosine terms and minimizes an error criterion that balances fitting the observed spectral samples with ensuring spectral smoothness (regularization).
  • Mel-Frequency Warping: To align the spectral representation with human auditory perception, the authors apply frequency warping. Specifically, they investigate mel-scale warping, which emphasizes lower frequencies, and a bilinear parametric warping function.
  • MRCC: By combining RCC with mel-frequency warping, the authors derive MRCC. They experimentally determine the optimal warping parameters and regularization constants for MRCC to achieve the best reconstruction quality.
  • Phase Estimation: For HMM-based TTS, the original phase information is not transmitted. The paper proposes estimating the phase from the MRCC amplitude parameters using a minimal phase spectrum reconstruction.
  • HMM-based TTS System: The MRCC parameters are then used within a standard HMM-based TTS framework (specifically, the Nitech-NAIST HMM-Based Speech Synthesis System) for spectral and excitation (logF0) modeling, along with Global Variance (GV).

Datasets

  • Training and Testing: The experiments were conducted using a standard 16 kHz US English female voice dataset (CMU Arctic, specifically the "slt" voice).
  • Data Split: The dataset was split into training and testing sets. 20 recordings were excluded from training and used for testing.
  • Evaluation: For objective quality evaluation, 25 male and 25 female sentences were analyzed. Subjective evaluations involved 10 subjects listening to 33 sentence pairs.

Key Insights and Ideas

  • MRCC for Sinusoidal Models: The paper demonstrates that MRCC is a superior parameterization for sinusoidal speech models in HMM-based TTS compared to traditional MGC-LSP. This is because MRCC is designed to capture the continuous spectral envelope more effectively, which is crucial for accurate sinusoidal reconstruction.
  • Importance of Warping and Regularization: The choice of frequency warping (mel-scale being optimal in this case) and the regularization constant significantly impacts the quality of the reconstructed speech. Too little or too much warping, or an inappropriate regularization, can lead to unstable spectral behavior or increased reconstruction errors.
  • Sinusoidal Synthesis Advantage: The results strongly suggest that sinusoidal waveform synthesis, even when using parameters derived from MGC-LSP, offers better quality than simpler source-filter methods. However, the MRCC-based sinusoidal synthesis further elevates this quality.
  • Spectral Detail Preservation: Spectrogram comparisons visually show that MRCC-based synthesis conveys more spectral detail, reducing the "spectral smearing" often seen with MGC-LSP.

Key Results

  • Subjective Quality Improvement: The proposed MRCC-based sinusoidal synthesis system significantly outperformed both the baseline HTS synthesis (using MLSA filter) and the MGC-LSP-based sinusoidal synthesis in subjective preference tests (p<0.000001).
  • Preference for MRCC: The MRCC-based system was also statistically preferred over the MGC-LSP-based sinusoidal synthesis (p<0.001).
  • Optimal Settings: Mel-scale warping with a regularization constant of λ = 2e-4 was found to provide the best results for the MRCC parameterization.
  • Objective Quality: While not the primary focus, objective quality evaluations (PESQ) also supported the superiority of the MRCC approach.

Limitations and Future Work

  • Phase Modeling: The authors acknowledge that they did not focus on improving the excitation/phase modeling, which they believe is essential for further enhancing synthesized speech quality.
  • Advanced Source-Filter Schemes: The paper notes that more advanced source-filter schemes (like STRAIGHT) were not evaluated, leaving room for comparison.
  • Excitation/Phase Improvement: Future work could explore more sophisticated methods for estimating and synthesizing the excitation and phase components of the speech signal.

Summary updated

BibTeX
@inproceedings{shechtman2010sinusoidal,
  author = {Slava Shechtman and Alex Sorin},
  title = {{Sinusoidal model parameterization for HMM-based TTS system}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure