TL;DR
This paper explores how the "Tonal Center of Gravity" (TCoG), a measure of the average pitch over an accented syllable, can better capture perceptual distinctions in speech prosody than traditional methods focusing on specific pitch turning points. They demonstrate that a simplified, binary representation of pitch contours, when used to calculate TCoG, is surprisingly effective for both production and perception tasks, suggesting a path towards more naturalistic synthetic speech variability.
What Problem Does This Paper Address?
Prosody, the rhythm and intonation of speech, is crucial for conveying meaning, emphasis, and phrasing. In speech synthesis, creating natural-sounding prosody is a significant challenge. Traditional approaches often rely on identifying specific "pitch turning points" (like the peak or the start of a pitch change) in the fundamental frequency (F0) contour to characterize intonational contrasts. However, these turning points can be unreliable due to various acoustic factors. This paper investigates whether a more holistic measure, the Tonal Center of Gravity (TCoG), can serve as a more robust and perceptually relevant parameter for understanding and generating prosodic variations, particularly in distinguishing between different types of pitch accents.
Methods
The core of the paper revolves around the Tonal Center of Gravity (TCoG) metric. The TCoG is calculated as a weighted average of F0 values over a specific time window, typically an accented syllable. The paper introduces a novel approach by applying a sigmoid warping function to the F0 contour before calculating the TCoG. This sigmoid function effectively creates a binary representation of the pitch contour: high pitch values contribute significantly to the TCoG calculation (weight ≈ 1), while low pitch values contribute very little (weight ≈ 0). This simplified TCoG, termed TCOGs, is then compared against the original TCoG and traditional turning point (TP) models.
Two experiments were conducted:
- Production Experiment: Subjects produced utterances designed to elicit either "Incredulity" or "Uncertainty" intonation patterns. The TCoG and TCOGs models were used to classify these productions and compared to a TP-only model.
- Perception Experiment: Synthesized stimuli with systematically varied pitch contour shapes and alignments were presented to listeners. Logistic regression models using TCoG and TCOGs as predictors were used to assess their ability to predict listener judgments.
Datasets
The paper does not introduce new datasets but rather uses elicited speech data from their own production experiment and synthesized stimuli for their perception experiment. The elicited data involved 6 subjects repeating target sentences in specific discourse contexts to elicit "Incredulity" and "Uncertainty" intonation patterns, resulting in 116 recorded productions. The perception experiment used 49 synthesized stimuli created by manipulating pitch contour shapes and alignments.
Key Insights and Ideas
- TCoG as a Robust Prosodic Feature: The TCoG proves to be a more effective classifier of pitch accent types than traditional turning point analysis, especially in the face of acoustic variability.
- The Power of Simplification (TCOGs): Surprisingly, a simplified TCoG (TCOGs) derived from a binary representation of the F0 contour performs as well as, and sometimes slightly better than, the original TCoG. This suggests that the precise shape of the F0 contour might be less perceptually important than the general range and timing of higher pitch values.
- Perceptual Equivalence of Diverse Contours: The TCOGs model implies that many auditorily distinct F0 contours can be functionally equivalent if they share the same F0 range and midpoint intercepts, leading to the same TCoG. This is a crucial insight for generating naturalistic variability in synthetic speech.
- Focus on Higher Pitch: The sigmoid warping function highlights that listeners may be more sensitive to the higher portions of pitch contours, potentially downplaying the perceptual significance of lower pitch movements.
Key Results
- Production Study: The TCOGs model achieved a classification accuracy of 94.3% for distinguishing between "Incredulity" and "Uncertainty" contours, slightly outperforming the original TCoG (92.5%) and significantly better than a TP-only model (not explicitly quantified but implied to be worse).
- Perception Study: Both TCoG models (TCoG and TCOGs) performed better than a turning-point-only model. The TCOGs model showed a slight improvement in predictive power (Nagelkerke r² of .443) compared to the TCoG model (.436), with both being statistically significant.
Limitations and Future Work
- Limitations: The authors acknowledge that while the TCOGs model is effective, the precise reasons for the improved performance of the simplified binary weights are still under investigation. They also note that the current work focuses on specific types of pitch accents and may not generalize to all prosodic phenomena.
- Future Work: The paper suggests that the finding of functional equivalence between diverse F0 contours offers a direct path to injecting naturalistic variability into synthetic speech. By generating contours that map to the same TCoG but differ in shape, synthetic speech can achieve more varied and less monotonous prosody without sacrificing perceptual accuracy of accent type. Further research could explore the specific phonetic characteristics that listeners are sensitive to and how these can be leveraged for more realistic synthesis.
Summary updated