A database design for a TTS synthesis system using lexical diphones

Tanya Lambert, Andrew BreenInterspeech 2004

TL;DR

This paper introduces a novel approach to designing speech synthesis databases by focusing on "lexical diphones" – units that include syllabic stress information. This method leads to smaller, more efficient databases that produce more natural-sounding synthesized speech compared to traditional phonemic-based approaches.

What Problem Does This Paper Address?

Building effective Text-to-Speech (TTS) synthesis systems hinges on having comprehensive speech databases. Traditional approaches often rely on collecting vast amounts of text and speech data to cover all possible phonemes and their combinations (like phonemic diphones or triphones). However, this can lead to astronomically large and costly databases. Furthermore, simply covering all phonemic units doesn't guarantee natural-sounding speech, as the prosodic variations (like stress and intonation) are crucial. The authors argue that existing databases, even those aiming for broad coverage, often fail to capture important phonological phenomena of English, particularly those related to stress. They propose a more efficient and effective database design that prioritizes units with inherent stress information.

Methods

The core of the paper is the proposal of a new database design centered around lexical diphones. Unlike phonemic diphones (which just represent the transition between two sounds), lexical diphones incorporate the stress of the syllables they belong to. This is achieved by analyzing text and speech data, annotating syllables with stress levels (primary, secondary, or unstressed), and then extracting these stress-annotated diphones.

To build this database, the authors employ a multi-pronged approach:

  • Phonetic Dictionaries: Used as a starting point to understand potential units.
  • Continuous Speech and Text: Analyzed to capture real-world usage and variations.
  • Phonological Phenomena Research: Incorporating known English phonological rules and exceptions.

A key algorithmic contribution is the application of multiple set cover algorithms. These algorithms are used to select the most informative sentences from various text sources (including specialized English usage like technical texts and names) to ensure optimal coverage of the desired lexical diphones and triphones within a smaller corpus. Manual corrections and additions are also performed to ensure coverage of specific phonological events and stress patterns.

Datasets

The paper utilizes several datasets for analysis and comparison:

  • Phonetic Dictionary: A large dictionary (approx. 300,000 orthographic entries) was used to analyze the distribution of lexical and phonemic diphones and triphones.
  • Various Text Genres: Samples of continuous text from different genres (e.g., literary works, newspapers, technical texts, names) were analyzed to understand diphone distribution in real-world usage.
  • Existing TTS Databases for Comparison:
    • Messiah: Used by BT's Laureate TTS system (279 sentences).
  • Arctic: Used by the Festival Speech Synthesis System (1132 sentences).
  • TIMIT-SI: (1890 sentences).
  • Newly Introduced Benchmark: The authors created their own core text corpus of 540 sentences, designed specifically for TTS synthesis, which forms the basis of their proposed database.

Key Insights and Ideas

  • Lexical Stress is Key: The most significant insight is that simply covering phonemes isn't enough for natural speech. Incorporating lexical stress into the basic units (diphones) dramatically improves the ability to capture prosodic variations and leads to more perceptually pleasing synthesized speech.
  • Smaller Databases Can Be Better: By focusing on stress-annotated units and using intelligent selection algorithms, the authors demonstrate that a smaller, well-designed database can offer better coverage of important phonological features than much larger, conventionally built databases.
  • Variability of Diphones: The study highlights that juncture diphones (at word boundaries) are more variable than diphones within words, and their stress patterns are particularly important to capture.
  • Genre-Specific Diphone Distribution: The frequency distribution of lexical diphones and triphones varies significantly across different text genres, underscoring the need for diverse text sources in database design.

Key Results

  • Improved Unit Coverage: The authors' database (540 sentences) showed a higher concentration of lexical diphones and triphones compared to the Messiah database (279 sentences), even though the latter is larger.
  • Perceptual Preference: Subjective evaluations showed listeners preferred speech synthesized from the authors' database over speech from the Messiah database. The Mean Opinion Score (MOS) for their database was 3.70 (SDev 0.81), compared to 2.74 (SDev 0.98) for the Messiah database.
  • Statistical Significance: The differences in lexical coverage between their database and Messiah were statistically significant (p<0.0000 for both diphones and triphones).
  • Unit Selection Efficiency: When synthesizing speech, their database led to a higher percentage of lexical units being selected (61%) compared to the Messiah database (50%).

Limitations and Future Work

  • Limitations: The paper acknowledges that while their database is smaller and more efficient, it doesn't aim to cover every single prosodic unit. The focus is on units that are perceptually significant.
  • Future Work: The authors plan to further extend their core database to achieve even broader coverage of phonological units. They aim to continue focusing on phonological features that have a significant perceptual impact on synthesized speech.

Summary updated

BibTeX
@inproceedings{lambert2004database,
  author = {Tanya Lambert and Andrew Breen},
  title = {{A database design for a TTS synthesis system using lexical diphones}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure