A hybrid approach to segmentation of speech using group delay processing and HMM based embedded reestimation

S. Aswin Shanmugam, Hema MurthyInterspeech 2014

TL;DR

This paper introduces a novel hybrid approach to speech segmentation that combines signal processing (group delay) with statistical modeling (HMMs) to achieve more accurate phone and syllable boundaries. By iteratively refining segmentation using both methods, they improve the performance of text-to-speech (TTS) systems, leading to more natural-sounding synthesized speech.

What Problem Does This Paper Address?

Accurate segmentation of speech into its constituent units (phones and syllables) is crucial for many speech processing applications, especially for training high-quality text-to-speech (TTS) systems. Traditional methods, like those based on Hidden Markov Models (HMMs) with embedded reestimation and forced alignment, often struggle to produce precise boundaries. This is partly because HMMs don't inherently consider boundary proximity during training, and phone transitions can be obscured by coarticulation. While syllable boundaries are generally more distinct due to lower energy between them, purely signal processing-based methods can introduce spurious boundaries without textual context. This paper aims to bridge this gap by creating a synergistic approach that leverages the strengths of both signal processing and statistical modeling for more robust and accurate segmentation.

Methods

The core of the proposed method is a hybrid approach that iteratively refines speech segmentation. It starts with a standard HMM-based segmentation process, which includes flat-start initialization, embedded training, and forced Viterbi alignment. However, this initial segmentation often suffers from durational errors and imprecise boundaries.

To address this, the paper incorporates group delay processing of short-time energy (STE). This signal processing technique is used to identify potential syllable boundaries by detecting regions of low acoustic energy. While group delay segmentation can be accurate, it can also produce spurious boundaries without textual information.

The hybrid method then intelligently combines these two approaches:

  1. Syllable Boundary Correction: Boundaries identified by the group delay method are used to correct the HMM-based syllable boundaries, particularly when the HMM boundaries are inaccurate or when the syllable doesn't end with certain phonetic classes (like nasals or fricatives).
  2. Syllable-Level Embedded Training: The speech data is then spliced at these refined syllable boundaries. Monophone HMM models are re-trained using embedded Baum-Welch reestimation specifically on these syllable-level segments. This makes the models more robust as they are trained on shorter, more focused speech units.
  3. Phone-Level Segmentation: With the refined syllable models, forced alignment is performed on the syllable waveforms to obtain phone-level segmentation within each syllable.
  4. Iterative Refinement: The process can be iterated, with the newly obtained phone boundaries used to further refine the syllable boundaries and re-estimate models.

This iterative synergy allows the statistical models to benefit from the precise acoustic cues provided by signal processing, and vice-versa, leading to improved segmentation at both syllable and phone levels.

Datasets

The experiments were conducted on a dataset of 730 utterances of Tamil data spoken by a single native female speaker. The paper mentions a phoneme set of 35 phonemes for Tamil. No new benchmark datasets were introduced; the focus was on improving segmentation on existing data.

Key Insights and Ideas

  • Synergy is Key: The most significant insight is that combining signal processing (group delay) and statistical modeling (HMMs) in an iterative manner yields better results than either method alone. Signal processing provides accurate acoustic cues, while HMMs provide robustness and context.
  • Syllable-Level Training Boosts Robustness: Restricting HMM reestimation to syllable-level segments significantly improves the robustness of the monophone models, as they are trained on more consistent units.
  • Group Delay as a Boundary Corrector: The group delay method, despite its potential for spurious boundaries, proves effective in correcting and refining HMM-based syllable boundaries, especially in challenging cases.
  • Improved TTS Quality: The improved segmentation directly translates to better performance in text-to-speech systems, indicating that accurate segmentation is a critical bottleneck for natural speech synthesis.

Key Results

  • Improved Acoustic Likelihood: The hybrid method led to an increase in the average log probability per frame from -74.16 (HMM) to -73.17 (Hybrid), indicating better acoustic modeling. This improvement was observed across various phoneme categories, including vowels, consonants, nasals, and fricatives.
  • Reduced Word Error Rate (WER) in TTS: When used to build HTS systems, the hybrid segmentation resulted in a significantly lower WER:
  • HTS - Syllable (Hybrid Segmentation): 7.07% (vs. 11.11% for HMM Segmentation)
    • HTS - Phone (Hybrid Segmentation): 1.01% (vs. 4.04% for HMM Segmentation)
  • Subjective Evaluation (Pair Comparison): Pair comparison tests showed a strong preference for synthesized speech from the hybrid method, with system "A" (hybrid) preferred over system "B" (HMM) in 77.5% of cases, indicating more natural-sounding speech.

Limitations and Future Work

  • Limitations:
    • The group delay segmentation, while useful for correction, can still introduce spurious boundaries, especially when syllables start or end with certain phonetic classes (e.g., semi-vowels, fricatives, nasals). The paper notes that the hybrid approach attempts to mitigate this by using the HMM context.
  • The current approach relies on a single native female speaker for the Tamil dataset. Generalizing to different speakers, accents, and languages might require further adaptation.
  • Future Work:
    • The authors suggest that the proposed hybrid approach can be extended to other languages and applications requiring accurate speech segmentation.
  • Further exploration of how to best integrate signal processing cues with statistical models for even more robust segmentation remains an open area.

Summary updated

BibTeX
@inproceedings{shanmugam2014hybrid,
  author = {S. Aswin Shanmugam and Hema Murthy},
  title = {{A hybrid approach to segmentation of speech using group delay processing and HMM based embedded reestimation}},
  booktitle = {Interspeech},
  year = {2014},
}
Figure