A comparison of methods for speaker-dependent pronunciation tuning for text-to-speech synthesis

Gabriel Webster, Tina Burrows, Katherine KnillInterspeech 2005

TL;DR

This paper tackles the challenge of making text-to-speech (TTS) systems sound natural by adapting generic pronunciations to individual speakers. They compare three automatic methods – N-grams, decision trees, and transformation-based learning (TBL) – finding that TBL significantly reduces pronunciation errors, making TTS voices more speaker-specific and natural-sounding.

What Problem Does This Paper Address?

Modern text-to-speech (TTS) systems often rely on pre-recorded speech units. To synthesize new speech, these systems need to generate phonetic pronunciations for input text and then select corresponding speech units. A key challenge is ensuring that the synthesized pronunciation style matches the style of the original recordings in the speech database. This is especially tricky because different speakers have distinct speech characteristics. To support multiple speakers, TTS systems need a way to "tune" generic, speaker-independent pronunciations to match a specific speaker's voice. This process, known as speaker-dependent pronunciation tuning, aims to bridge the gap between the system's predicted pronunciation and how a particular speaker would actually say it. The paper focuses on improving this tuning process by exploring automatic methods that can learn these speaker-specific variations.

Methods

The authors investigate three automatic methods for speaker-dependent pronunciation tuning:

  • Word N-grams: This approach uses statistical models (N-grams) trained on the speech database to identify common pronunciation variations for specific word contexts. When the input text matches a learned N-gram, the system uses the database-derived pronunciation instead of its default one. They experimented with unigrams, bigrams, and trigrams, and even considered word affixes to improve generalization.
  • Decision Trees: Decision trees are used to learn a mapping from input features (like surrounding phones and orthography) to an output class (a modified pronunciation). Each system phone is processed by a decision tree to determine its speaker-specific pronunciation. The features include a window of surrounding phones, whether those phones are vowels, and a clustered orthographic representation of the word.
  • Transformation-Based Learning (TBL): TBL is presented as a more powerful technique than decision trees. It automatically learns a set of transformation rules that convert a sequence of system-generated phones into a speaker-specific sequence. These rules are learned iteratively, with each rule improving accuracy over the previous ones. The rules can consider various features, including surrounding phones and orthography, and can model more complex interactions than decision trees.

Datasets

The experiments were conducted using Toshiba's American English TTS system and a speech database consisting of 2,437 sentences spoken by a single female American English speaker. The data was split into 90% for training and 10% for evaluation. Further divisions of the training data were used for model development and parameter tuning.

Key Insights and Ideas

  • TBL's Power: The paper strongly suggests that TBL's ability to learn complex, sequential transformations makes it superior for this task compared to N-grams and decision trees.
  • Context is Crucial: All methods that incorporate contextual information (surrounding phones, word affixes) perform better than simpler approaches, highlighting the importance of coarticulation and speaker-specific variations that depend on neighboring sounds.
  • Orthography Matters: Incorporating orthographic information (how words are spelled) into the models, especially in the decision tree approach, proved beneficial for capturing pronunciation nuances.
  • Generalization is Key: The N-gram approach's limitation to words seen in training data significantly impacts its performance on unseen words, underscoring the need for methods that can generalize.

Key Results

Transformation-Based Learning (TBL) achieved the best results, reducing the phone error rate (PER) by 26% and the word error rate (WER) by 23% compared to the baseline (unmodified system pronunciations). Decision trees offered a significant improvement as well, with a 21% PER reduction and 18% WER reduction. The N-gram approach showed the smallest improvement, with a 13% PER reduction and 14% WER reduction. Notably, TBL also showed the largest improvement on unseen test words, demonstrating its superior generalization capabilities.

Limitations and Future Work

The authors acknowledge that the N-gram model's performance is limited by its inability to handle words not present in the training data. They also note that while TBL is more powerful, its computational efficiency for a single phone depends on the implementation, and a fast implementation was not used in their experiments.

Future work suggested includes extending these methods to different speakers and languages. They also propose enriching the feature set available to TBL to include more linguistic information, such as lexical stress, phrasal stress, pausing, and part-of-speech tags, which could lead to further improvements in speaker-dependent pronunciation tuning accuracy.

Summary updated

BibTeX
@inproceedings{webster2005comparison,
  author = {Gabriel Webster and Tina Burrows and Katherine Knill},
  title = {{A comparison of methods for speaker-dependent pronunciation tuning for text-to-speech synthesis}},
  booktitle = {Interspeech},
  year = {2005},
}
Figure