A decision tree-based clustering approach to state definition in an excitation modeling framework for HMM-based speech synthesis

Ranniery Maia, Tomoki Toda, Keiichi Tokuda, Shinsuke Sakai, Satoshi NakamuraInterspeech 2009

TL;DR

This paper introduces a novel decision tree-based clustering method to define states for modeling the excitation signal in HMM-based speech synthesis. By directly clustering residual signals using a residual likelihood criterion, this approach improves upon previous methods that relied on mel-cepstral coefficients, leading to more accurate and natural-sounding synthesized speech.

What Problem Does This Paper Address?

Generating natural-sounding speech with Hidden Markov Models (HMMs) for text-to-speech (TTS) synthesis requires accurately modeling the excitation signal, which represents the fundamental sound source. Traditional HMM-based TTS often uses a simple model where voiced segments are generated from pulse trains and unvoiced segments from white noise. More advanced methods, like the one proposed by Maia et al. in [1], aim to improve this by modeling the excitation signal using state-dependent filters applied to pulse trains and noise.

However, a crucial challenge in these advanced excitation models is defining the states that these filters belong to. Previous approaches either relied on phonetic decision trees (which indirectly relate spectral features to excitation) or used a hybrid method that merged states based on residual likelihood but still started from mel-cepstral coefficient decision trees. The authors argue that these indirect methods are approximations and that directly clustering residual signals based on their own likelihood would be more effective. This paper tackles the problem of how to define these excitation model states directly and effectively using residual signal properties.

Methods

The core contribution of this paper is a decision tree-based clustering algorithm for defining excitation model states. Unlike prior methods that used mel-cepstral coefficients to guide state definition, this approach directly uses the residual signal's likelihood as the criterion for splitting and merging states.

Here's a breakdown of the method:

  • Excitation Model Framework: The paper builds upon an existing excitation modeling framework [1] where voiced and unvoiced filters (Hv(z) and Hu(z)) are used to shape pulse trains and white noise, respectively, to generate the final excitation signal. These filters are state-dependent, meaning different HMM states have different filter characteristics.
  • Direct Residual Clustering: The proposed algorithm constructs a decision tree where each node represents a cluster of residual segments.
  • Top-Down Splitting: The process starts with all residual segments in a single cluster. At each step, the algorithm considers splitting a cluster into two child clusters. The split is determined by a "question" (related to full context features, though the paper notes phonetic features are often selected). The criterion for choosing the best split is the maximum likelihood increment of the residual signal, calculated using the formula in equation (13).
  • Clustering Criterion: The likelihood of a residual segment under a given filter state is modeled assuming the unvoiced excitation component is a Gaussian process. The log-likelihood calculation, particularly for the unvoiced filter gain (K), is derived and used as the basis for the splitting criterion.
  • Filter Estimation: While the primary goal is state definition, voiced filter coefficients are estimated using a least-squares approach, and unvoiced filter coefficients (and their gain) are determined via linear prediction on the unvoiced excitation signal. This estimation is done iteratively during the tree construction process.
  • Stopping Criterion: Tree growth is controlled using the Minimum Description Length (MDL) criterion, balancing model complexity with likelihood improvement.

Datasets

The experiments were conducted using the ATR503 database, which was recorded from a female Japanese speaker. This database was used to train an HMM-based synthesizer, and residual signals were extracted by inverse filtering using the mel-cepstral coefficients from this training. The paper does not introduce a new benchmark dataset.

Key Insights and Ideas

  • Direct Residual Likelihood is Superior: The paper strongly advocates for and demonstrates that directly using residual signal likelihood for clustering is more effective than relying on mel-cepstral coefficients. This is because residual signals are the direct target of the excitation modeling filters, making them a more relevant feature for defining states.
  • Phonetic Features Still Relevant: Despite the algorithm using full context questions, the generated decision trees predominantly selected questions related to current and left phonetic context. This suggests that phonetic information is inherently linked to excitation characteristics, validating some aspects of previous phonetic tree-based approaches.
  • Voiced Segments Get More Granularity: The clustering process resulted in more detailed sub-clustering for voiced segments compared to unvoiced ones. This is intuitively appealing, as voiced sounds often have richer and more complex excitation patterns.
  • Unified State Definition: The proposed method offers a more unified approach to state definition by directly optimizing for the excitation model's performance, rather than using spectral features as a proxy.

Key Results

  • The proposed algorithm successfully clustered residual segments, leading to the definition of excitation model states.
  • A total of 97 clusters (terminal nodes) were created across all HMM states.
  • The central HMM states were clustered with more detail, which is expected as they typically contain more speech segments.
  • The generated decision trees showed a preference for phonetic context questions, aligning with findings from previous phonetic tree-based methods.
  • Figure 4 shows impulse responses of the derived voiced filters, indicating that the algorithm achieved convergence for most filters, suggesting successful state configuration.

Limitations and Future Work

  • Segmentation and Pitch Marking: The authors acknowledge that segmentation and pitch marking errors, which are external to their algorithm, might have contributed to some noisy impulse responses observed in the results. These issues are considered beyond the scope of the current paper.
  • Unified State Clustering and Filter Optimization: The current algorithm focuses on state definition, with filter estimation being a part of the process. The authors suggest that a future direction would be to develop a method that simultaneously optimizes state clustering and filter parameters for a more integrated approach.

Summary updated

BibTeX
@inproceedings{maia2009decision,
  author = {Ranniery Maia and Tomoki Toda and Keiichi Tokuda and Shinsuke Sakai and Satoshi Nakamura},
  title = {{A decision tree-based clustering approach to state definition in an excitation modeling framework for HMM-based speech synthesis}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure