A bi-lingual Mandarin-to-taiwanese text-to-speech system

Min-Siong Liang, Ke-Chun Chuang, Rhuei-Cheng Yang, Yuang-Chin Chiang, Ren-Yuan LyuInterspeech 2005

TL;DR

This paper presents a novel bi-lingual Text-to-Speech (TTS) system that converts Mandarin Chinese text into Taiwanese speech. It addresses the challenge of limited linguistic resources for Taiwanese by leveraging bilingual lexicon information and a three-module architecture: text analysis, prosody, and waveform synthesis. The system achieves promising accuracy in text analysis and tone-sandhi, paving the way for applications like educational tools for language learners.

What Problem Does This Paper Address?

Taiwan is a multilingual society where Mandarin and Taiwanese are widely spoken, but linguistic resources for Taiwanese are scarce, especially for written forms and educational tools. While Mandarin TTS systems are abundant, Taiwanese TTS systems are rare. A significant challenge is the lack of a standardized written form for Taiwanese, leading to multiple writing systems and pronunciation variations (literate vs. oral). Furthermore, Taiwanese is a tonal language, and accurately capturing tone sandhi rules is crucial for natural-sounding speech synthesis. This paper aims to bridge this gap by developing a bi-lingual TTS system that can translate Mandarin text into Taiwanese speech, thereby facilitating the learning and preservation of the Taiwanese language.

Methods

The proposed TTS system is built upon a three-module architecture:

  • Text Analysis Module: This module handles the conversion of Mandarin text into a phonetic representation of Taiwanese. It involves:
  • Word Segmentation and Translation: A sequential maximal-length matching algorithm is used to segment Mandarin sentences into words and then translate them word-by-word into Taiwanese using a bilingual lexicon.
  • Labeling (Morpheme-to-Phoneme): A two-stage strategy is employed to handle multiple pronunciations for Taiwanese words, prioritizing oral pronunciation and then using a Viterbi search with pronunciation frequencies.
  • Tone Sandhi Rules: The system incorporates specific rules for tone sandhi in Taiwanese, which is a tonal language. This includes handling lexical tones and their transformations based on context.
  • ForPA (Formosa Phonetic Alphabet): A new phonetic symbol set, ForPA, is proposed to address the inadequacy of existing systems (like Pinyin and Zhuyin) for transcribing Taiwanese and Hakka, aiming for a more suitable representation for multilingual speech data labeling.
  • Digit Sequence Normalization: Rules are defined to normalize the pronunciation of digit sequences, considering both literate and oral pronunciations based on their position.
  • Prosody Module: This module generates prosodic information, including tone contours and duration, for the synthesized speech.
  • Waveform Synthesis Module: This module synthesizes the speech signal. The paper mentions the use of TD-PSOLA (Time-Domain Pitch Synchronous Overlap Add) to modify prosodic features of selected units, aiming for more natural-sounding speech by adjusting pitch and duration.

Datasets

  • Mandarin-Taiwanese Bilingual Lexicon: This lexicon is a key resource, containing Mandarin words and their corresponding Taiwanese pronunciations (both literate and oral) transcribed in ForPA. Statistics on the number of pronunciations per word are provided.
  • News Articles: A dataset of news articles was collected from the internet for evaluation. Sentences longer than 20 characters were removed.
  • Manual Transcription: A set of 200 sentences from the news corpus was manually transcribed by two Taiwanese linguistics experts for comparison with the system's automatic transcription.

Key Insights and Ideas

  • The Need for a Unified Phonetic System: The paper highlights the confusion arising from multiple phonetic transcription systems for Taiwanese and proposes ForPA as a more suitable alternative for multilingual speech analysis.
  • Leveraging Bilingual Lexicons for Translation: The core idea of using a bilingual lexicon to bridge the gap between Mandarin and Taiwanese text for TTS is central to the system's design.
  • Handling Pronunciation Ambiguity: The two-stage strategy for selecting pronunciations (oral first, then literate) and the Viterbi search for optimal pronunciation are crucial for dealing with the inherent variability in Taiwanese.
  • Importance of Tone Sandhi: The paper emphasizes the complexity and necessity of accurately modeling tone sandhi rules for natural Taiwanese speech synthesis. The "tone sandhi sailboat" and rules for stop tones illustrate this.
  • Potential for Educational Tools: The system's ability to convert Mandarin text to Taiwanese speech opens doors for creating valuable educational resources, such as talking electronic lexicons.

Key Results

  • Text Analysis and Tone-Sandhi Accuracy: The system achieved approximately 90% labeling accuracy and 65% tone-sandhi accuracy in experiments.
  • Word Segmentation and Transfer Accuracy: When tone-sandhi was not considered, the system achieved high accuracy rates of over 97% for word segmentation and transfer.
  • Limitations in Names and Out-of-Vocabulary: The paper notes that most errors occur in names and out-of-vocabulary words.
  • Acceptable Tone-Sandhi Accuracy: Despite the lower accuracy for tone-sandhi, it is deemed acceptable due to the uniform tone-sandhi rules in Taiwanese.

Limitations and Future Work

  • Tone-Sandhi Improvement: The authors acknowledge that improving tone-sandhi accuracy is a key area for future work to achieve more accurate speech synthesis.
  • Handling Names and Out-of-Vocabulary Words: Further research is needed to address the errors encountered with names and words not present in the lexicon.
  • Development of Educational Tools: The paper suggests building a talking electronic lexicon and an interactive phonetic alphabet learning tool using the developed TTS system.
  • Spreading ForPA: The authors express the need to spread the use of ForPA through interactive learning tools.

Summary updated

BibTeX
@inproceedings{liang2005bi,
  author = {Min-Siong Liang and Ke-Chun Chuang and Rhuei-Cheng Yang and Yuang-Chin Chiang and Ren-Yuan Lyu},
  title = {{A bi-lingual Mandarin-to-taiwanese text-to-speech system}},
  booktitle = {Interspeech},
  year = {2005},
}
Figure