Guided-TTS: Text-to-Speech with Untranscribed Speech

Heeseung Kim, Sungwon Kim, Sungroh Yoon

Abstract

We propose Guided-TTS, a high-quality text-to-speech (TTS) model that does not require any transcript of target speaker using classifier guidance. Guided-TTS combines an unconditional diffusion probabilistic model with a separately trained phoneme classifier for classifier guidance. Our unconditional diffusion model learns to generate speech without any context from untranscribed speech data. For TTS synthesis, we guide the generative process of the diffusion model with a phoneme classifier trained on a large-scale speech recognition dataset. We present a norm-based scaling method that reduces the pronunciation errors of classifier guidance in Guided-TTS. We show that Guided-TTS achieves a performance comparable to that of the state-of-the-art TTS model, Grad-TTS, without any transcript for LJSpeech. We further demonstrate that Guided-TTS performs well on diverse datasets including a long-form untranscribed dataset.

Introduction

Building a high-quality Text-to-Speech (TTS) model usually requires a "gold standard" dataset: hours of high-quality audio paired with perfectly accurate text transcripts. While the internet is full of audio (podcasts, audiobooks, lectures), most of it is untranscribed.

Existing methods to use this "wild" data usually involve using an Automatic Speech Recognition (ASR) model to generate "noisy" transcripts first, but these errors propagate into the TTS model. Guided-TTS aims to bridge this gap by learning the distribution of a speaker's voice unconditionally and then using a general-purpose phoneme recognizer to steer that voice to say specific words.

Methodology

The architecture of Guided-TTS is built on Denoising Diffusion Probabilistic Models (DDPM). Instead of training a model to map text directly to audio, the authors break the task into two parts:

  1. Unconditional DDPM: This model is trained on random chunks of untranscribed audio from a target speaker. It learns how to generate mel-spectrograms that "sound" like that speaker, but it doesn't know what it's saying.
  2. Classifier Guidance: Borrowing a technique from computer vision, the authors use a separate Phoneme Classifier (trained on large, public transcribed datasets like LibriSpeech). During the "denoising" process of the diffusion model, this classifier looks at the noisy output and provides a gradient—essentially a "nudge"—to move the generated audio toward the desired phonemes.
Overview of the Guided-TTS architecture, showing the interaction between the unconditional DDPM, phoneme classifier, and duration predictor.

Norm-Based Guidance

The authors discovered a major hurdle: as the diffusion model gets closer to producing a clean output, its internal "score" (the gradient of the data distribution) becomes much larger than the classifier's gradient. This causes the model to ignore the text and produce random speech-like sounds. To fix this, they introduced Norm-based Guidance, which scales the classifier's gradient to match the norm of the diffusion score, ensuring the "guidance" remains effective throughout the entire generation process.

Experimental Results

Guided-TTS was evaluated across several benchmarks, including LJSpeech, Hi-Fi TTS, and the Blizzard 2013 audiobook dataset.

Performance comparison of Guided-TTS against ground truth and baseline models across multiple datasets.

Key findings include:

  • Performance Parity: On the LJSpeech dataset, Guided-TTS achieved a Mean Opinion Score (MOS) of 4.25, matching the performance of Grad-TTS (a state-of-the-art conditional model) even though Guided-TTS did not use any transcripts from the LJSpeech dataset.
  • Accuracy: The Character Error Rate (CER) was 1.03%, significantly lower than baseline methods that try to use ASR-generated transcripts (which hit 1.16% or higher).
  • Robustness to "Wild" Data: Because the DDPM is unconditional, it doesn't need to know where a sentence starts or ends. The model performed robustly on the unsegmented Blizzard dataset, proving it can learn from raw, unsegmented audio clips.

Conclusion

Guided-TTS demonstrates that high-quality speech synthesis is possible without speaker-specific transcripts by decoupling content from style. While inference speed remains a challenge due to the iterative nature of diffusion models, the system's ability to utilize untranscribed data significantly lowers the barrier for creating high-quality TTS for new speakers. Future work may focus on scaling the phoneme classifier with even larger ASR datasets to further reduce pronunciation errors and improve handling of diverse speaking styles.

Summary updated

Abstract

We propose Guided-TTS, a high-quality text-to-speech (TTS) model that does not require any transcript of target speaker using classifier guidance. Guided-TTS combines an unconditional diffusion probabilistic model with a separately trained phoneme classifier for classifier guidance. Our unconditional diffusion model learns to generate speech without any context from untranscribed speech data. For TTS synthesis, we guide the generative process of the diffusion model with a phoneme classifier trained on a large-scale speech recognition dataset. We present a norm-based scaling method that reduces the pronunciation errors of classifier guidance in Guided-TTS. We show that Guided-TTS achieves a performance comparable to that of the state-of-the-art TTS model, Grad-TTS, without any transcript for LJSpeech. We further demonstrate that Guided-TTS performs well on diverse datasets including a long-form untranscribed dataset.
BibTeX
@inproceedings{kim2022guided,
  author = {Heeseung Kim and Sungwon Kim and Sungroh Yoon},
  title = {{Guided-TTS: Text-to-Speech with Untranscribed Speech}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=W3SxRos66mT},
}
Figure