Overview of SHRC-Ginkgo speech synthesis system for Blizzard Challenge 2013

Yansuo Yu, Fengyun Zhu, Xiangang Li, Yi Liu, Jun Zou, Yuning Yang, Guilin Yang, Ziye Fan, Xihong WuBlizzard Challenge 2013 · OpenReview

TL;DR

This paper presents the SHRC-Ginkgo speech synthesis system, which leverages lightly-supervised acoustic model training and rich syntactic context to build high-quality voices from large, roughly labeled audiobook corpora. The system achieves state-of-the-art results in speaker similarity and naturalness for the Blizzard Challenge 2013, demonstrating the effectiveness of its novel data preparation and modeling techniques.

What Problem Does This Paper Address?

Creating high-quality Text-to-Speech (TTS) systems typically requires large amounts of clean speech data with accurate transcriptions. However, real-world speech corpora, especially those derived from audiobooks, often suffer from imperfect labeling, inconsistent audio quality, and variations in speaking style. Manually cleaning and transcribing such massive datasets is prohibitively time-consuming and expensive. This paper tackles the challenge of building a robust TTS system from such "messy" data, specifically for the Blizzard Challenge 2013, which involved large audiobook corpora with limited or no pre-existing accurate transcriptions.

Methods

The SHRC-Ginkgo system employs a unit selection-based approach for speech synthesis. Its key innovations lie in the data preparation and acoustic modeling stages:

  • Lightly-Supervised Acoustic Model Training for Data Selection: Instead of relying on perfect transcriptions, the system uses speech recognition with a transcription-related language model to identify and select "clean" speech segments with accurate text from the roughly labeled corpora. This process involves calculating Word Error Rates (WER) to filter out sentences with transcription errors.
  • Syntactic Parsing for Acoustic Model Refinement: Traditional TTS systems often use prosodic structures to model acoustic variations. This paper proposes using rich syntactic context derived from automatic syntactic parsing to refine acoustic models. This approach captures more linguistic information and can automatically label large corpora, avoiding manual prosodic annotation.
  • Fast Acoustic Model Training: To handle the memory and computational burden of training on hundreds of hours of speech, a fast training method is developed that maintains acoustic model accuracy.

Datasets

The system was developed and evaluated using datasets provided for the Blizzard Challenge 2013 English tasks (EH1 and EH2).

  • EH1: Approximately 300 hours of chapter-sized MP3 files from an audiobook corpus, with unsegmented audio and no provided text. Participants had to obtain text from the web and align it.
  • EH2: Approximately 19 hours of non-compressed WAV files, also from an audiobook corpus, with segmented audio and accompanying text (though participants could also obtain text from the web).

Both datasets were from a single female speaker and provided by The Voice Factory.

Key Insights and Ideas

  • Syntactic Context is Powerful: Replacing traditional prosodic structure with rich syntactic context significantly improves acoustic modeling, especially for large corpora. This suggests that understanding sentence structure provides more valuable information for synthesis than just intonation patterns.
  • Leveraging Imperfect Data: The system demonstrates that even with imperfectly labeled data, effective TTS systems can be built by employing intelligent data selection and cleaning techniques like lightly-supervised speech recognition.
  • Scalability Matters: The development of a fast training method is crucial for handling the massive datasets characteristic of modern TTS challenges, enabling the use of more data without prohibitive computational costs.

Key Results

Subjective evaluation results from the Blizzard Challenge 2013 indicate:

  • Speaker Similarity: The SHRC-Ginkgo system (identified as 'M') achieved the best similarity to the original speaker for both EH1 and EH2 tasks, with statistically significant differences compared to other participant systems.
  • Naturalness: The system performed best among all participant systems (excluding the natural speech benchmark) in terms of naturalness for both EH1 and EH2, again with statistically significant improvements.
  • Intelligibility: The system achieved the 3rd and 4th lowest word error rates (WER) for EH1 and EH2, respectively, demonstrating good intelligibility.
  • Paragraph Test: The system was rated as the best for EH1 and top-3 for EH2 across various aspects of paragraph-level synthesis, including overall impression, pleasantness, and intonation.

Limitations and Future Work

The authors acknowledge that some important problems remain to be solved:

  • Emotion Expression: The current system does not explicitly handle emotion expression for different roles within the audiobook.
  • Fast Training for Large Corpora: While a fast training method was developed, further optimization for acoustic model training on very large corpora is still needed.
  • Prosodic Modeling: The paper suggests that while syntactic context is beneficial, further integration of prosodic information might yield even better results.

Future work could focus on incorporating emotional prosody, further optimizing training efficiency, and exploring more advanced modeling techniques.

Summary updated

Abstract

This paper introduces the SHRC-Ginkgo speech synthesis system for Blizzard Challenge 2013. A unit selection based approach is adopted to develop our speech synthesis system using audiobook speech corpus. Aiming at roughly labeled corpora with several hundred hours of speech, our system adopts lightly-supervised acoustic model training of speech recognition to select clean speech data with accurate text. Moreover, rich syntactic contexts instead of prosodic structure are utilized to refine traditional acoustic models. Through automatic syntactic parsing, this way can also help to label the corpora of several tens or even hundreds of hours automatically, thus avoiding manually prosodic annotation with time-consuming and expensive effort. In order to solve the problems of memory space expansion and running time burden for acoustic model training of large-scale corpora, a fast training method, which can ensure the accuracy of acoustic model, is realized. Subjective evaluation results show that our system performs well in almost all evaluation tests, especially in the case of large-scale corpora.
BibTeX
@inproceedings{yu2013overview,
  author = {Yansuo Yu and Fengyun Zhu and Xiangang Li and Yi Liu and Jun Zou and Yuning Yang and Guilin Yang and Ziye Fan and Xihong Wu},
  title = {{Overview of SHRC-Ginkgo speech synthesis system for Blizzard Challenge 2013}},
  booktitle = {Blizzard Challenge},
  year = {2013},
  url = {https://openreview.net/forum?id=nEmNzHNlBI},
}
Figure