Accurate pitch marking for prosodic modification of speech segments

Thomas Ewender, Beat PfisterInterspeech 2010

TL;DR

This paper introduces a novel pitch marking algorithm that improves speech synthesis quality by more accurately identifying glottal closure instants (GCIs). By intelligently combining short-term energy contours with fundamental wave analysis, the method overcomes limitations of traditional approaches, leading to more natural-sounding synthesized speech.

What Problem Does This Paper Address?

Accurate pitch marking is crucial for prosodic modification techniques like TD-PSOLA, which are used in concatenative speech synthesis to alter pitch and speaking rate. The quality of synthesized speech heavily relies on precisely segmenting speech into double-period units, ideally starting at glottal closure instants (GCIs). However, directly measuring GCIs is difficult as the glottal airflow signal is rarely available. Existing methods for estimating pitch marks (which approximate GCIs) from the speech signal often struggle with noisy or irregular speech segments, leading to artifacts like pitch doubling or jitter in synthesized speech. This paper aims to address these robustness issues by proposing a more accurate and reliable pitch marking algorithm.

Methods

The proposed pitch marking algorithm takes a hybrid approach, intelligently switching between two primary features:

  • Short-Term Energy Contour: This is the primary feature. The algorithm leverages the idea that the maximum of the short-term energy contour often corresponds to the GCI. The short-term energy is computed using a Hamming window whose size is adapted to the local signal period.
  • Fundamental Wave: When the short-term energy contour proves unreliable (e.g., due to noise, pitch doubling, or broad peaks), the algorithm falls back to using the fundamental wave. This is computed by low-pass filtering the speech signal.

The core innovation lies in the criteria-based switching between these two features. The algorithm first attempts to use short-term energy peaks. It then rigorously evaluates the reliability of these energy peaks based on several criteria, including peak prominence, proximity to adjacent valleys/peaks, and peak shape. If an energy peak is deemed unreliable, the algorithm then assesses the reliability of the fundamental wave using its own set of criteria (amplitude, valley/peak positions, regularity). If the fundamental wave is deemed reliable, it's used to set pitch marks, often by interpolating or extrapolating from existing reliable marks. This combined approach ensures that pitch marks are placed accurately even in challenging acoustic conditions.

Datasets

The paper mentions using "a set of 9 voices that were known to be problematic for pitch marking" and "four different voices, one male and one female Dutch voice, one female German and one male American English voice" for evaluation. These voices were recorded in studio quality. No new datasets were introduced.

Key Insights and Ideas

  • Energy Peaks as Primary Indicators: The paper validates the intuition that short-term energy peaks are good indicators of GCIs, especially in clean speech.
  • The Need for Robust Fallback: Critically, the authors demonstrate that relying solely on energy peaks is insufficient. They meticulously identify and categorize scenarios where energy peaks fail (pitch doubling, jitter, spurious peaks due to noise).
  • Intelligent Feature Switching: The most significant insight is the development of a robust mechanism to switch between energy and fundamental wave features based on reliability criteria. This adaptive strategy is key to achieving high accuracy across diverse speech conditions.
  • Fundamental Wave as a Complement, Not a Replacement: The paper emphasizes that the fundamental wave alone is not sufficient for pitch marking due to phase ambiguity. Its strength lies in its ability to complement energy-based methods in problematic regions.
  • Subjective Evaluation Matters: The authors chose to evaluate their method not just on objective metrics but also on the perceived quality of synthesized speech, highlighting the practical impact of accurate pitch marking.

Key Results

The paper reports that their proposed pitch marking algorithm "clearly improves the quality of synthesised speech" when used with a TD-PSOLA based text-to-speech system. Specifically, synthesized speech using their method showed:

  • Clear improvement for two male voices, with less reverberation and a less raspy sound.
  • Significant improvement for a bassy male American English voice, making it sound more sonorous and less "frizzled."
  • Isolated improvements for female voices in mixed or irregular speech segments.
  • The method effectively eliminates artifacts caused by other pitch markers (like Praat's cross-correlation algorithm) in synthetic speech.

Limitations and Future Work

  • Limitations: The paper acknowledges that the criteria for determining unreliable energy peaks and fundamental wave segments were manually determined using a set of 9 voices and are "virtually voice-independent." However, the "noise thresholds" are dynamically estimated. The paper also notes that in rare cases where no reliable short-term energy peak is found, the algorithm selects a peak that "fulfil most of the reliability criteria," implying potential for further refinement.
  • Future Work: Not explicitly detailed, but the success in improving synthesized speech quality suggests further exploration in applying this robust pitch marking to various TTS systems and potentially exploring more advanced feature extraction or switching mechanisms.

Summary updated

BibTeX
@inproceedings{ewender2010accurate,
  author = {Thomas Ewender and Beat Pfister},
  title = {{Accurate pitch marking for prosodic modification of speech segments}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure