TL;DR
This paper investigates how pitch and energy patterns in speech can help us understand and predict disfluencies (like "ums" and "ahs") in European Portuguese lectures. They found that the way pitch and energy change around a disfluency, rather than just the disfluency itself, is a strong indicator, suggesting speakers use prosodic "contrast" to signal these speech interruptions.
What Problem Does This Paper Address?
Disfluencies are a natural part of spontaneous speech, but they pose significant challenges for speech processing technologies like automatic speech recognition (ASR) and text-to-speech (TTS) synthesis. To make these systems more robust and natural, researchers need to understand the acoustic and prosodic properties of disfluencies and how they interact with the surrounding speech. While previous work has explored this, there's a need for more detailed analysis, especially for specific languages like European Portuguese, and to understand if prosodic cues are used consistently across different types of disfluencies and speakers. This paper aims to characterize these prosodic patterns in university lectures to improve our understanding and predictive models of disfluencies.
Methods
The researchers analyzed a corpus of European Portuguese university lectures. They extracted prosodic features like pitch (fundamental frequency, F0) and energy, along with phonetic and duration information. Their core method involved analyzing these prosodic features not only within the disfluent segments themselves but also in the words immediately preceding (reparandum) and following (repair) the disfluency. They specifically looked at the slopes of pitch and energy within these segments, as well as the differences in these slopes between adjacent segments. Statistical tests (Kruskall-Wallis) were used to determine the significance of observed trends. They also investigated whether these patterns varied across different types of disfluencies (e.g., filled pauses, repetitions, substitutions) and across different speakers.
Datasets
The study utilized the "Lectra corpus," a collection of 75 hours of transcribed university lectures in European Portuguese. A subset of this corpus, specifically the training subset, was used for the analysis. The paper notes that 27 hours were orthographically transcribed, and speech recognition was used for force-aligned transcriptions. The total alignment error for the subset was 10.2%.
Key Insights and Ideas
- Prosodic Contrast is Key: The most significant finding is that speakers seem to use a strategy of "prosodic contrast" rather than "parallelism" around disfluencies. This means the pitch and energy patterns change noticeably between the disfluency and the surrounding speech, rather than mirroring each other.
- Context Matters More Than the Disfluency Itself: The prosodic features of the words immediately before and after the disfluency are highly informative, often more so than the disfluency segment alone. This suggests speakers use these adjacent contexts to signal the disfluency.
- Disfluency Type and Degree of Contrast: While pitch and energy generally increase in the repair region, the degree of this increase varies by disfluency type. Filled pauses, for instance, show the highest pitch reset, similar to a full stop, while repetitions are notable for their energy slopes.
- Speaker Consistency: The observed prosodic trends were largely consistent across six out of the seven speakers, suggesting these are general patterns in European Portuguese lectures rather than highly idiosyncratic behaviors.
- Efficiency in Speech: The authors hypothesize that this use of contrast in adjacent contexts is an economical way for speakers to signal disfluencies, helping listeners process information efficiently within a short memory interval.
Key Results
- Significant Slope Differences: Pitch and energy slopes were found to be significantly different within the words immediately before and after disfluencies (p < 0.001).
- Pitch and Energy Resets: Pitch and energy generally increase from the disfluency region to the repair region. This reset was significant even when excluding silent pauses, which occurred in 99% of cases.
- Filled Pauses Lead: Filled pauses showed the highest pitch reset, significantly different from other disfluency types.
- Repetitions and Energy: Repetitions showed significantly different energy slopes compared to other types, with the highest energy slope within the repair.
- Speaker 6 "Likeability": Interestingly, the speaker identified as most "likeable" by annotators (Speaker 6) exhibited the highest energy slope within the repair and a considerable pitch increase, correlating with traits associated with charismatic speech.
Limitations and Future Work
- Limited Speaker Range: While six out of seven speakers showed consistent trends, the analysis was based on a small number of speakers, limiting broad generalizations.
- Focus on Lectures: The findings are specific to the domain of university lectures. Future work could explore other speech genres and languages to see if these prosodic patterns hold.
- Teaching Skills Hypothesis: The authors suggest that the economical and contrastive ways speakers manage disfluencies might be linked to teaching skills, implying speakers choose informative yet less disruptive cues. Future research could investigate this further.
- Cross-linguistic Studies: The paper calls for comparable studies in other languages and domains to further understand the universality and specificity of these prosodic disfluency cues.
Summary updated