TL;DR
This paper investigates how voice quality, specifically the glottal source parameter $R_d$, can be used to signal prominence in synthetic speech without altering fundamental frequency ($f_0$). They found that manipulating $R_d$ effectively creates perceived prominence, and interestingly, this manipulation extends beyond the focal syllable to influence the entire utterance, suggesting a more holistic approach to prosody modeling in TTS.
What Problem Does This Paper Address?
Creating natural-sounding synthetic speech, particularly in terms of prosody (rhythm, stress, and intonation), remains a significant challenge. Prominence, the perceived emphasis on certain syllables or words, is a key aspect of prosody that signals importance and meaning. Traditional methods for generating prominence in Text-to-Speech (TTS) systems often rely on manipulating fundamental frequency ($f_0$) or duration. However, this paper explores an alternative: using variations in voice quality, specifically the glottal source parameter $R_d$, to signal prominence. The authors aim to understand how different baseline voice qualities (breathy, modal, tense) affect the perception of prominence when $R_d$ is manipulated, and whether this manipulation is localized to the focal syllable or has a broader impact on the utterance. This research is particularly relevant for developing synthetic voices for Irish dialects, where a limited set of control parameters is desired.
Methods
The core of this research lies in a novel perception experiment that uses a manipulation task rather than a traditional rating task. Participants were presented with synthetic mini-dialogues designed to elicit narrow focus. They were then asked to actively manipulate an unknown parameter (which was, in fact, $R_d$) in the synthetic utterances to make them sound like natural responses. This "user-driven" approach allows researchers to observe what acoustic manipulations listeners deem perceptually salient for prominence.
The synthetic speech was generated using a glottal source model where $R_d$ was the primary control parameter, and $f_0$ was kept constant. Three baseline voice qualities (breathy, modal, tense) were used, each with a different baseline $R_d$ value. The $R_d$ parameter was manipulated within specific ranges for each voice quality. A user interface was developed to allow participants to adjust $R_d$ for individual words by dragging blocks up and down, which controlled a scaling factor applied to the baseline $R_d$ contour.
Datasets
The stimuli for the perception test were based on a recording of an all-voiced sentence: "We were away a day ago." This sentence was chosen for its potentially accentable syllables (WAY and DAY) and was recorded by a male Irish English speaker. The authors then created synthetic versions of this sentence for the experiment. No new benchmark datasets were introduced; rather, existing recordings were used as the basis for synthetic stimuli.
Key Insights and Ideas
- Voice Quality as a Prominence Cue: The study strongly supports the idea that voice quality, specifically $R_d$, can effectively signal prominence in synthetic speech, even without $f_0$ variation. This opens up new avenues for prosody modeling in TTS.
- Holistic Utterance Influence: Prominence manipulation via $R_d$ isn't confined to the focal syllable. Participants adjusted $R_d$ in pre- and post-focal material as well, suggesting that prominence signaling involves a broader contour change across the entire utterance, not just localized emphasis.
- Direction of Manipulation: Across all voice qualities, participants consistently lowered $R_d$ (leading to tenser phonation) in focal syllables to signal prominence. This aligns with previous findings.
- Logarithmic Scaling for Salience: While linear scaling of $R_d$ showed differences in manipulation magnitude across voice qualities, a logarithmic scale revealed that these differences were not statistically significant. This suggests that logarithmic manipulation of $R_d$ might be more perceptually meaningful for achieving prominence across different voice qualities.
- User-Driven Approach Value: The manipulation task proved valuable, providing insights into what listeners perceive as natural and prominent, offering a potentially more accurate representation of perceptual salience compared to rating pre-defined stimuli.
Key Results
- Participants successfully manipulated $R_d$ to create perceptually salient prominence.
- The direction of $R_d$ manipulation in focal syllables was consistently towards lower values (tenser phonation) across breathy, modal, and tense voice qualities.
- The magnitude of $R_d$ manipulation was significantly less for tense voice compared to breathy and modal voice when using a linear scale. However, this difference was not statistically significant when using a logarithmic scale.
- The location of the focal syllable within the utterance (e.g., early vs. late) did not significantly affect the magnitude of $R_d$ excursions.
Limitations and Future Work
- Language Specificity: The study's findings on the direction of $R_d$ manipulation (tenser phonation for prominence) are based on English and Irish speakers. The authors acknowledge that this trend might not be universal, citing Finnish data where focal syllables were characterized by laxer/breathier phonation. Further research is needed to explore language-specific variations.
- Interaction of $f_0$ and $R_d$: While this study deliberately excluded $f_0$ variation to isolate the effect of $R_d$, future work could investigate the interplay between $f_0$ and $R_d$ manipulation for even richer prosodic expression.
- Deaccentuation Role: The paper suggests that the increase in $R_d$ (laxer phonation) in pre- and post-focal material (deaccentuation) plays a role in signaling prominence. However, the relative importance of this deaccentuation versus the focal accentuation itself was not directly assessed and warrants further investigation.
- Logarithmic Manipulation in TTS: The finding that logarithmic scaling of $R_d$ might be more perceptually appropriate suggests that TTS systems should consider implementing this for more natural prosody generation.
- User Interface Refinement: The user-driven manipulation task is a promising avenue. Future work could explore how to refine such interfaces for more intuitive control and to gather richer data on listener preferences.
Summary updated