Improved training of excitation for HMM-based parametric speech synthesis

Yoshinori Shiga, Tomoki Toda, Shinsuke Sakai, Hisashi KawaiInterspeech 2010

TL;DR

This paper introduces a novel method for training the unvoiced excitation component in HMM-based speech synthesis. By separating the periodic and non-periodic components of the residual signal, they achieve "contamination-free" targets for training the unvoiced filter, resulting in significantly less noisy synthetic speech compared to conventional methods.

What Problem Does This Paper Address?

Parametric speech synthesis, particularly using Hidden Markov Models (HMMs), offers flexibility but often struggles with unnatural speech quality, especially in the "roughness" or noisiness of the output. A key source of this issue lies in the parametric excitation model, which generates the sound source for the vocal tract. The conventional approach to training the unvoiced excitation filter, which handles non-periodic sounds like fricatives, suffers from a problem: it uses the residual signal after estimating the voiced excitation as a target. This residual is contaminated by errors from the voiced excitation estimation, leading to an overestimated unvoiced filter and, consequently, noisy synthetic speech. The paper aims to solve this by finding a way to train the unvoiced filter using cleaner targets.

Methods

The core innovation lies in the improved training scheme for the unvoiced excitation filter. Instead of using the residual signal directly after voiced excitation estimation, the authors propose a "contamination-free" approach.

  1. Periodic/Non-Periodic Decomposition: The key step is to decompose the residual signal e(n) into its periodic and non-periodic components. This is achieved using a "periodic component estimator." The periodic component is modeled as a sum of harmonics with piecewise linear amplitude and frequency, while the remaining non-periodic component is treated as the "clean" target for unvoiced filter training.
  2. Unvoiced Filter Training: The unvoiced filter coefficients Hu(z) and its gain Ks are then determined using linear prediction analysis on this extracted non-periodic component, specifically for segments tagged with a particular HMM state.
  3. Tree-Based State Definition: Similar to the voiced filter training, a decision-tree-based approach is used to define states for the unvoiced filter, allowing for state-dependent filter coefficients. This means a separate tree is constructed for unvoiced filter training.

The overall training process iteratively optimizes both voiced and unvoiced filters, along with the excitation signal itself, to minimize the mean squared error between the synthesized excitation and the residual.

Datasets

The paper uses a dataset of 4014 English utterances by a British male speaker, recorded by Phonetic Arts Ltd. and released for the Blizzard Challenge 2010. The evaluation also uses 20 sentences from the Blizzard Challenge 2009 test set (news and novel categories).

Key Insights and Ideas

  • Contamination is the Root Cause: The paper clearly identifies that errors in voiced excitation estimation directly contaminate the targets for unvoiced excitation training, leading to a fundamental problem in conventional methods.
  • Decomposition as a Solution: Separating the periodic and non-periodic components of the residual is a clever way to isolate the true unvoiced excitation signal, providing cleaner training targets.
  • Improved Spectral Characteristics: The improved training results in unvoiced filter responses with significantly lower spectral energy in the low-frequency range, which directly translates to less perceived noise in the synthesized speech.
  • HPF as a Band-Aid: While a high-pass filter (HPF) is often used to reduce unvoiced noise, the paper shows it's a partial solution that can remove useful high-frequency unvoiced information. The improved method reduces the need for such aggressive filtering.

Key Results

Subjective evaluation results show a strong preference for speech synthesized using the improved training method.

  • When no HPF is applied to the synthesized unvoiced excitation, listeners preferred the improved method in 97% of cases.
  • Even when an HPF is applied to both conventional and improved methods, the improved method still yields slightly better quality, though the difference is less pronounced.
  • Qualitatively, the noise from the conventional method is described as "rather stationary," while the noise from the improved method (when it occurs) is "intermittent" and "segmental," suggesting it might arise at state boundaries.

Limitations and Future Work

  • Perceptible Noise at State Boundaries: The authors acknowledge that the unvoiced filter response changes abruptly between states, which can still cause perceptible noise, especially since the current model doesn't use dynamic features for the filter response.
  • Unvoiced Excitation Power: The power of the voiced excitation cannot be determined within the current framework.
  • Future Work: The authors suggest incorporating dynamic features into the filter response to mitigate state-boundary noise and accounting for the amplitude of optimized impulses in determining the voiced excitation power.

Summary updated

BibTeX
@inproceedings{shiga2010improved,
  author = {Yoshinori Shiga and Tomoki Toda and Shinsuke Sakai and Hisashi Kawai},
  title = {{Improved training of excitation for HMM-based parametric speech synthesis}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure