TL;DR
This paper introduces ProsoProm, a novel system for automatically detecting gradual prominence in speech, moving beyond binary classifications. By combining four acoustic features, ProsoProm assigns a continuous prominence score to each syllable, achieving a strong agreement with human annotations.
What Problem Does This Paper Address?
Traditionally, identifying prominence in speech (the emphasis given to certain syllables or words) has been treated as a binary decision: either a unit is prominent or it isn't. However, human perception of prominence is more nuanced, existing on a spectrum. Existing automatic systems largely mirror this binary view, failing to capture the subtle variations in how prominent a syllable can be. This paper addresses the challenge of creating an automatic system that can quantify this gradual nature of prominence, providing a more fine-grained analysis of speech prosody.
Methods
The ProsoProm system utilizes a cumulative approach, combining four key acoustic features:
- Relative Pitch: How a syllable's pitch compares to its surrounding context.
- Relative Duration: How a syllable's length compares to its surrounding context.
- Pitch Movement: The change in pitch within a syllable.
- Duration of Following Pause: The length of the silent gap after a syllable.
These features are normalized and then combined into a single "promCUM" score. This cumulative score is then transformed into a 0-4 integer scale using a generalized sigmoid function, where 0 represents non-prominent and 4 represents extra-prominent. The system was trained and evaluated on a manually annotated corpus.
Datasets
The study used data extracted from the PFC database. This included:
- 398 words from 22 sentences read by 3 Parisian speakers.
- Approximately 200 seconds of monologue speech per speaker from free interviews. In total, the corpus comprised 18 minutes and 41 seconds of speech, containing 3669 annotated syllables. The manual annotations were performed by three experts from the paper's author team.
Key Insights and Ideas
- Prominence is Gradual: The core insight is that prominence isn't an on/off switch but a continuous scale, and automatic systems should reflect this.
- Cumulative Feature Integration: Combining multiple acoustic features into a single score is more effective than relying on individual feature thresholds, as it can capture subtle prominences that might be missed otherwise.
- Importance of Pitch Movement Refinement: The way pitch movement is handled (specifically, favoring "positive movement" or rising tones) significantly impacts the accuracy of prominence detection.
- Relativization Span Matters: The context window used for normalizing features (e.g., comparing a syllable to its neighbors) needs careful tuning for optimal results.
Key Results
- The ProsoProm system achieved a Spearman's rho of 0.776 and a quadratic Kappa coefficient of 0.803 when compared to the median human annotation, indicating a very good agreement.
- This performance significantly outperformed a previous threshold-based method (Kappa of 0.61 vs. 0.67 for ProsoProm in a 2-class annotation) and a similar existing system (Analor, with a Spearman's rho of 0.694).
- Qualitative analysis revealed that the system sometimes missed very low prominences due to weak acoustic cues and that human annotators could be influenced by syntactic finality, even without strong acoustic evidence.
Limitations and Future Work
- Closed Test Evaluation: The authors acknowledge that the evaluation was performed in a "closed test" setting, meaning the same data was used for both optimization and evaluation. Future work should involve open testing or n-fold cross-validation.
- Sequential Optimization: The parameter tuning was sequential (span, then weights). A more robust, integrated optimization methodology is suggested.
- Further Evaluation: The authors recommend evaluating ProsoProm on diverse speech materials with varying content to assess its robustness.
- Hesitation Syllables: Future investigations could focus on better detecting final lowering syllables and optionally hesitation syllables.
Summary updated