Vocalic sandwich, a unit designed for unit selection TTS

Didier Cadic, Cédric Boidin, Christophe d'AlessandroInterspeech 2009

TL;DR

This paper introduces the "vocalic sandwich," a novel phonetic unit designed to improve the efficiency and quality of speech synthesis corpus design. By focusing on robust consonant-vowel-consonant structures, vocalic sandwiches offer a more compact and perceptually relevant unit for building high-quality TTS voices with less data.

What Problem Does This Paper Address?

Modern text-to-speech (TTS) systems often rely on concatenating pre-recorded speech segments. To achieve natural-sounding speech, these systems require large databases of recordings from a single speaker. However, creating these databases (corpora) is time-consuming and expensive. The challenge is to optimize the corpus design process to capture the necessary phonetic and prosodic variations with minimal recordings. Traditional units used in speech technology, like diphones or triphones, are not specifically tailored for this corpus design task, leading to inefficiencies. The authors aim to address this by proposing a new unit that better reflects the characteristics of concatenative TTS and its impact on speech quality.

Methods

The core contribution is the introduction of the "vocalic sandwich" unit. This unit is defined as a sequence of phonemes structured as S+ V S+, where S represents a set of "robust" phonemes (typically consonants and silence) that are less susceptible to splicing artifacts, and V represents a sequence of "fragile" phonemes (vowels and semi-vowels) that are more sensitive to concatenation issues. The idea is that by framing these fragile phonemes with robust ones, the unit becomes more suitable for concatenation and less prone to audible glitches at the boundaries.

The paper then compares the statistical properties of vocalic sandwiches (both context-independent and context-dependent versions) against traditional units like phone n-grams, syllable n-grams, and word n-grams. This objective comparison includes metrics like mean length, number of distinct units, 80% coverage, density, overlap ratio, and entropy.

Furthermore, a perceptual evaluation was conducted. Synthesized utterances using different unit types for corpus design were rated by human listeners on a Mean Opinion Score (MOS) scale. The authors then correlated these MOS scores with the coverage rates of various unit types to assess their predictive power for perceived speech quality.

Datasets

The statistical analysis was performed on a French textual corpus of 179,000 words from the domain of Interactive Voice Response (IVR) applications.

For the perceptual evaluation, 144 distinct utterances were synthesized using the Orange Labs diphone unit selection TTS system. These utterances were drawn from two application domains: IVR applications and movie subtitles. The synthesized utterances were created using two versions of the TTS voice: one with a full acoustic inventory (approximately 3 hours of speech) and one with a reduced acoustic inventory (30% of the full inventory, approximately 1 hour of speech). This resulted in a total of 288 synthesized utterances.

Key Insights and Ideas

  • Vocalic sandwiches are more "compact": Statistically, vocalic sandwiches show a more compact distribution of units compared to phone n-grams of similar length, meaning fewer distinct units are needed to cover a significant portion of the corpus. This is crucial for efficient corpus design.
  • Perceptual relevance: The definition of vocalic sandwiches is rooted in the perceptual challenges of concatenative TTS, particularly around vowels and semi-vowels. This suggests they are inherently more aligned with perceived speech quality.
  • Predictive power for quality: The study found that the coverage rate of vocalic sandwich 2-grams (two consecutive vocalic sandwiches) correlated most strongly with human judgments of speech quality (MOS scores). This indicates their effectiveness in predicting how good a synthesized utterance will sound based on the corpus used.
  • Context matters, but less than expected: While context-dependent units generally showed higher dispersion than their context-independent counterparts, the difference was not as pronounced as anticipated, especially for the expressive voice used. This suggests that for certain types of speech, the inherent structure of the unit might be more critical than fine-grained contextual information.

Key Results

  • Objective comparison: Sandwich 1-grams and 2-grams exhibited a good balance between unit length and compactness (measured by 80% coverage and entropy) compared to traditional units. They also had a lower overlap ratio, offering flexibility in corpus construction.
  • Perceptual correlation: The highest correlation coefficient between MOS scores and unit coverage rates was observed for context-independent sandwich 2-grams (0.57). Context-dependent sandwich 2-grams also performed well (0.53). This was significantly higher than the correlation with the TTS engine's unit selection cost (-0.48).
  • Optimal unit length: The results suggest that units around 5-6 phonemes in length (like sandwich 2-grams) might be optimal for predicting speech quality, which is longer than commonly used units like phone 3-grams.

Limitations and Future Work

  • Expressive voice bias: The perceptual evaluation was conducted using an expressive female French voice. The authors acknowledge that the results might differ for more neutral voices, and suggest reproducing the experiment with voices of various speaking styles.
  • Language specificity: The vocalic sandwich unit was developed and tested for French. Further investigation is needed to validate its relevance for other languages.
  • Application scope: While promising for corpus design, the authors suggest using vocalic sandwiches for actual applications like database reduction and exploring their potential in other areas of speech synthesis.
  • Contextual information: The authors note that the limited set of contexts (13 for French neutral speech) might have influenced the performance of context-dependent units. Future work could explore different contextual strategies.

Summary updated

BibTeX
@inproceedings{cadic2009vocalic,
  author = {Didier Cadic and Cédric Boidin and Christophe d'Alessandro},
  title = {{Vocalic sandwich, a unit designed for unit selection TTS}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure