Acoustic Analysis of Prosodic Features in Natural versus Synthesized Speech Samples from YourTTS and SYNTACC Models

Julio Cesar Galdino, Gustavo Araújo, Arnaldo Cândido, Miguel Oliveira, Moacir Antonelli Ponti, Sandra Maria Aluísio2024 · DOI

Abstract

This study presents an acoustic analysis of prosodic features in both natural and synthesized speech samples, using two state-of-the-art speech synthesis models: YourTTS and SYNTACC. By analyzing spontaneous speech data, the duration of intonational units and syllables produced by these models was compared. The findings reveal that both models generate speech with significantly shorter and less variable durations of intonational units and syllables compared to natural speech. These results highlight the differences in syllable duration and speech rate between synthesized and natural speech, emphasizing the need for more refined prosodic metrics to accurately assess the quality of synthesized speech.

Introduction

While modern Text-to-Speech (TTS) models can mimic a person's voice with startling accuracy, they often struggle to capture the underlying rhythm and "soul" of human speech. This phenomenon is largely due to prosody—the complex interplay of rhythm, stress, and intonation. Current TTS evaluation metrics often prioritize subjective Mean Opinion Scores (MOS) or objective measures like Speaker Embedding Cosine Similarity (SECS). While these metrics confirm if a voice sounds like a specific target, they fail to capture whether the timing and cadence are natural.

This research addresses this gap by performing a deep acoustic analysis of Brazilian Portuguese (BP), a language historically considered low-resource for TTS. The study specifically investigates whether modern models can replicate the complex timing of spontaneous human speech, which is inherently more variable than the "read speech" typically used for training.

Methodology

The researchers conducted an objective acoustic analysis comparing natural speech (Ground Truth) against samples generated by two distinct models:

  1. YourTTS: A zero-shot multi-speaker model based on the VITS architecture.
  2. SYNTACC: A model designed to handle multiple accents by "freezing" specific model weights to preserve regional linguistic variants.

Using Praat software and the AnalyseTier script, speech was segmented into Intonational Units (IUs)—chunks of speech with complete meaning—and specific syllable types (nuclear vs. non-nuclear). The analysis also accounted for silent and filled pauses. Statistical significance was determined using ANOVA and Tukey tests to compare duration and speech rate across the samples.

Datasets

The study utilized a post-processed version of the Museu da Pessoa (MuPe) Corpus (MuPe-v1). This dataset is unique because it contains spontaneous speech from life stories rather than scripted readings. It encompasses 90.25 hours of speech across 87,076 segments, featuring 86 speakers from 11 Brazilian states and two foreign countries. This diversity provides a rigorous test for the models' ability to handle natural hesitations and varied regional rhythms.

Key Results and Discussion

The analysis revealed several critical differences between natural and synthetic speech, particularly regarding speed and variability.

The "Robotic" Speed Trap

AI models tend to function as "speed-talkers." While humans naturally speak at an average rate of 6.67 syllables per second, the AI models pushed significantly higher, reaching between 9.3 and 9.6 syllables per second. This 40% increase in speed can lead to comprehension failures in human listeners.

Comparison of speech rate (syllables per second) across natural and synthetic speech.

Duration and Structural Markers

Natural speech (GT) exhibited an average Intonational Unit duration of 1,692 ms. In contrast, SYNTACC and YourTTS produced significantly shorter units at 1,194 ms and 1,256 ms, respectively. Furthermore, the models struggled with the structural use of silence; YourTTS failed to produce a single silent pause in the test set, whereas humans use pauses as essential markers to help listeners process information.

Comparative duration metrics for Ground Truth (GT) and synthesized models.

Rhythmic Rigidity

In natural speech, "nuclear" syllables are elongated (averaging 280 ms) to provide emphasis. While the models did make these syllables longer than non-nuclear ones, they did not stretch them nearly as much as humans (averaging only 147–158 ms). Additionally, the standard deviation for syllable duration in natural speech (76.4 ms) was nearly double that of the AI models (~36 ms), indicating that AI speech is much more monotonic and rhythmicly rigid.

Conclusion and Future Work

The study concludes that while modern TTS models are proficient at voice mimicry, they lack the temporal flexibility of human speech. Future research will expand this analysis to include fundamental frequency (F0/pitch) and intensity to provide a more holistic view of prosody. Additionally, the researchers plan to explore architectures like Non-Attentive Tacotron, which utilize dedicated duration predictors to better control the tempo and regional rhythmic identities of generated speech.

Summary updated

Abstract

This study presents an acoustic analysis of prosodic features in both natural and synthesized speech samples, using two state-of-the-art speech synthesis models: YourTTS and SYNTACC. By analyzing spontaneous speech data, the duration of intonational units and syllables produced by these models was compared. The findings reveal that both models generate speech with significantly shorter and less variable durations of intonational units and syllables compared to natural speech. These results highlight the differences in syllable duration and speech rate between synthesized and natural speech, emphasizing the need for more refined prosodic metrics to accurately assess the quality of synthesized speech.
BibTeX
@misc{galdino2024acoustic,
  author = {Julio Cesar Galdino and Gustavo Araújo and Arnaldo Cândido and Miguel Oliveira and Moacir Antonelli Ponti and Sandra Maria Aluísio},
  title = {{Acoustic Analysis of Prosodic Features in Natural versus Synthesized Speech Samples from YourTTS and SYNTACC Models}},
  year = {2024},
  doi = {10.5753/eniac.2024.245092},
}
Figure