Abstract
This study presents an acoustic analysis of prosodic features in both natural and synthesized speech samples, using two state-of-the-art speech synthesis models: YourTTS and SYNTACC. By analyzing spontaneous speech data, the duration of intonational units and syllables produced by these models was compared. The findings reveal that both models generate speech with significantly shorter and less variable durations of intonational units and syllables compared to natural speech. These results highlight the differences in syllable duration and speech rate between synthesized and natural speech, emphasizing the need for more refined prosodic metrics to accurately assess the quality of synthesized speech.
Introduction
While modern Text-to-Speech (TTS) models can mimic a person's voice with startling accuracy, they often struggle to capture the underlying rhythm and "soul" of human speech. This phenomenon is largely due to prosody—the complex interplay of rhythm, stress, and intonation. Current TTS evaluation metrics often prioritize subjective Mean Opinion Scores (MOS) or objective measures like Speaker Embedding Cosine Similarity (SECS). While these metrics confirm if a voice sounds like a specific target, they fail to capture whether the timing and cadence are natural.
This research addresses this gap by performing a deep acoustic analysis of Brazilian Portuguese (BP), a language historically considered low-resource for TTS. The study specifically investigates whether modern models can replicate the complex timing of spontaneous human speech, which is inherently more variable than the "read speech" typically used for training.
Methodology
The researchers conducted an objective acoustic analysis comparing natural speech (Ground Truth) against samples generated by two distinct models:
- YourTTS: A zero-shot multi-speaker model based on the VITS architecture.
- SYNTACC: A model designed to handle multiple accents by "freezing" specific model weights to preserve regional linguistic variants.
Using Praat software and the AnalyseTier script, speech was segmented into Intonational Units (IUs)—chunks of speech with complete meaning—and specific syllable types (nuclear vs. non-nuclear). The analysis also accounted for silent and filled pauses. Statistical significance was determined using ANOVA and Tukey tests to compare duration and speech rate across the samples.
Datasets
The study utilized a post-processed version of the Museu da Pessoa (MuPe) Corpus (MuPe-v1). This dataset is unique because it contains spontaneous speech from life stories rather than scripted readings. It encompasses 90.25 hours of speech across 87,076 segments, featuring 86 speakers from 11 Brazilian states and two foreign countries. This diversity provides a rigorous test for the models' ability to handle natural hesitations and varied regional rhythms.
Key Results and Discussion
The analysis revealed several critical differences between natural and synthetic speech, particularly regarding speed and variability.
The "Robotic" Speed Trap
AI models tend to function as "speed-talkers." While humans naturally speak at an average rate of 6.67 syllables per second, the AI models pushed significantly higher, reaching between 9.3 and 9.6 syllables per second. This 40% increase in speed can lead to comprehension failures in human listeners.
Duration and Structural Markers
Natural speech (GT) exhibited an average Intonational Unit duration of 1,692 ms. In contrast, SYNTACC and YourTTS produced significantly shorter units at 1,194 ms and 1,256 ms, respectively. Furthermore, the models struggled with the structural use of silence; YourTTS failed to produce a single silent pause in the test set, whereas humans use pauses as essential markers to help listeners process information.
Rhythmic Rigidity
In natural speech, "nuclear" syllables are elongated (averaging 280 ms) to provide emphasis. While the models did make these syllables longer than non-nuclear ones, they did not stretch them nearly as much as humans (averaging only 147–158 ms). Additionally, the standard deviation for syllable duration in natural speech (76.4 ms) was nearly double that of the AI models (~36 ms), indicating that AI speech is much more monotonic and rhythmicly rigid.
Conclusion and Future Work
The study concludes that while modern TTS models are proficient at voice mimicry, they lack the temporal flexibility of human speech. Future research will expand this analysis to include fundamental frequency (F0/pitch) and intensity to provide a more holistic view of prosody. Additionally, the researchers plan to explore architectures like Non-Attentive Tacotron, which utilize dedicated duration predictors to better control the tempo and regional rhythmic identities of generated speech.
Summary updated