A comparison of statistical methods and features for the prediction of prosodic structures

Qin Shi, Volker Fischer2004 · DOI

TL;DR

This paper tackles the challenge of predicting prosodic structures in speech, a crucial step for natural-sounding text-to-speech (TTS) systems. The authors compare dynamic programming (DP) and decision tree (DT) methods, finding that a hybrid DP+DT approach excels for word-level predictions, while DT alone is best for phrase-level predictions. They also demonstrate the transferability of these methods across different languages.

What Problem Does This Paper Address?

Creating natural-sounding synthetic speech is a complex task, and a key component is accurately predicting the rhythm and intonation, collectively known as prosody. Prosody influences how understandable and pleasant a synthesized voice is. Predicting prosodic structures (like where phrases begin and end) is a vital precursor to generating these prosodic features in text-to-speech (TTS) systems. While syntactic information plays a role, it doesn't fully determine prosody; speaker habits and other factors are also influential. This paper investigates how well different statistical methods can predict these prosodic boundaries, focusing on the trade-offs between accuracy, corpus size, and the features used.

Methods

The authors compare three statistical approaches for prosody structure prediction:

  • Dynamic Programming (DP): This method finds the optimal sequence of prosodic boundaries by considering global probabilities. It's good at finding the best overall path but might not fully leverage local context.
  • Decision Trees (DT): This approach uses a tree structure to make predictions based on local features. It's effective at capturing local context but might not be as globally optimal as DP.
  • Combined DP+DT: This hybrid approach aims to leverage the strengths of both. It uses DT to provide local probabilities that then adjust the scores within the DP search, effectively combining global optimization with local context awareness.

The prediction tasks involve identifying boundaries at different prosodic levels: prosodic word, prosodic phrase, and intonation phrase. They also explore the impact of various lexical features, such as part-of-speech tags, syllable counts, and punctuation.

Datasets

The primary dataset used is the Mandarin Chinese TTS corpus, containing approximately 22,000 carefully designed sentences. This corpus was manually annotated for prosodic structures.

For cross-language transfer experiments, they also used:

  • A Korean corpus of 6,600 sentences.
  • A German corpus of 5,500 sentences, annotated for both male and female speakers.

Key Insights and Ideas

  • Hybrid Power: Combining DP and DT methods offers a significant advantage, particularly for predicting prosodic word boundaries, outperforming either method alone.
  • Task-Specific Strengths: While DP+DT is great for local predictions (word boundaries), DT alone proves more effective for global predictions (phrase boundaries).
  • Corpus Size Matters (Mostly): Both DP and the combined DP+DT methods are sensitive to training data size, with accuracy increasing significantly as more data is added.
  • DT's Resilience: Decision trees are less sensitive to corpus size and can achieve good performance even with limited training data, making them a strong choice when data is scarce.
  • Feature Impact: Basic lexical features like POS tags and syllable counts are effective. However, the explored "distance to previous boundary" feature did not significantly improve prosodic phrase prediction accuracy in their experiments.
  • Language Transferability: The successful methods developed for Chinese can be effectively applied to Korean and German, suggesting a degree of universality in prosodic prediction strategies.

Key Results

  • For prosodic word boundary prediction, the DP+DT combination consistently achieved the highest recall and precision rates across various training corpus sizes, reaching up to 96.6% recall and 96.1% precision with the full 22K corpus.
  • For prosodic phrase boundary prediction, the DT method alone achieved the highest precision (88.8%) with the full 22K corpus, while DP+DT offered competitive recall (93.1%).
  • The "distance to previous boundary" feature, when added to DT for prosodic phrase prediction, showed minimal impact on accuracy (e.g., recall of 88.1% vs. 88.0% with the feature).
  • Cross-language experiments showed promising results: Korean phrase boundary prediction achieved recall/precision rates of 82.4%/82.5% (male speaker), and German achieved up to 81.53% recall and 87.34% precision (female speaker with punctuation features).

Limitations and Future Work

  • Distance Feature Ineffectiveness: The authors acknowledge that the "distance to previous boundary" feature did not yield the expected improvement in prosodic phrase prediction accuracy.
  • Future Directions: Future work will focus on exploring a wider range of lexical and statistical features, such as different POS tagging schemes and incorporating information content of words and phrases. They also aim to further investigate the transferability of methods to more languages and explore their application to intonation phrase prediction.

Summary updated

BibTeX
@misc{shi2004comparison,
  author = {Qin Shi and Volker Fischer},
  title = {{A comparison of statistical methods and features for the prediction of prosodic structures}},
  year = {2004},
  doi = {10.21437/interspeech.2004-464},
}
Figure