TL;DR
This paper tackles the challenge of modeling "focus" – the word that stands out in a sentence – by crowdsourcing annotations and building predictive models. They demonstrate that focus can be predicted from text alone, and that incorporating acoustic features significantly improves prediction accuracy, paving the way for more natural and context-aware speech synthesis.
What Problem Does This Paper Address?
When we speak, certain words naturally receive more emphasis or prominence, a phenomenon linguists call "focus." This focus isn't arbitrary; it often signals new or important information, helps disambiguate meaning, and is crucial for conveying the speaker's intent. While we intuitively understand focus, modeling it computationally, especially in the context of spoken language and discourse, is challenging. Current text-to-speech (TTS) systems often synthesize sentences in isolation, lacking the ability to adapt prosody based on preceding context, leading to robotic and unnatural-sounding speech. This paper aims to bridge this gap by analyzing how focus is perceived and produced in context, and by developing models to predict it.
Methods
The authors employed a multi-pronged approach:
- Crowdsourced Annotation: They created a corpus of sentence pairs (context + target sentence) and had human annotators identify the "focus" word in the target sentence. This was done for both text-only stimuli and isolated/contextualized speech recordings.
- Agreement Analysis: They analyzed the consistency of human annotations across different conditions (text vs. speech, with/without context) to understand the inherent ambiguity and predictability of focus.
- Text-Based Prediction: They trained a Support Vector Machine (SVM) classifier using lexical features (part-of-speech, word length, etc.) extracted from the text of the sentence and its context to predict focus words.
- Speech-Based Prediction: They used Classification and Regression Trees (CART) to model focus prediction using both lexical and acoustic features (like pitch, duration) from the speech signal.
- Contextual Prosody Analysis: They analyzed acoustic features of the same sentence spoken with and without context to understand how speakers adapt their prosody.
Datasets
- Custom Corpus: A corpus of approximately 1000 sentence pairs was created, derived from the Brown corpus. This corpus was balanced for genre and filtered for inappropriate material.
- Speech Recordings: A subset of 100 sentences from the corpus was recorded by a single female graduate student, both in isolation and with preceding context.
Key Insights and Ideas
- Focus is Predictable: Even with text alone, human annotators show significant agreement on focus words, suggesting a systematic underlying structure that can be modeled.
- Context Matters for Speech: While text-based focus prediction is feasible, incorporating acoustic features derived from speech significantly boosts prediction accuracy. This highlights the importance of prosody in conveying focus.
- Contextual Adaptation in Speech: Speakers demonstrably alter their prosodic patterns (e.g., pitch contours) when speaking a sentence in context compared to speaking it in isolation, aiming for coherence.
- Primary vs. Secondary Focus: The study identified distinct patterns for primary and secondary focus words, with primary focus often appearing earlier in the sentence and secondary focus later. Verbs were more likely to be secondary focus words.
Key Results
- Text-based Prediction: An SVM model achieved an accuracy of 0.44 and a Mean Reciprocal Rank (MRR) of 0.62 in predicting focus words from text alone.
- Speech-based Prediction: Incorporating acoustic features with lexical features improved the Mean Reciprocal Rank (MRR) for focus prediction in speech, though specific MRR values are not directly comparable across different metrics. The paper states that the "true focussed word is always within the top two model predictions" when acoustic features are used.
- Contextual Prosody: The study found that acoustic features like starting F0, ending F0, and maximum F0 values were consistently lower for sentences spoken in context compared to isolation.
- F0 Prediction: Predicting the maximum F0 value in a stimulus sentence showed a lower error (17.24 RMSE) when contextual features of the previous utterance's F0 were included, compared to prediction without context (18.48 RMSE).
Limitations and Future Work
- Limited Speech Data: The speech dataset was relatively small (100 sentences), limiting the scale of classification experiments.
- Single Speaker: Speech data was recorded from only one speaker, which might limit the generalizability of findings related to speaker-specific prosodic variations.
- Illustrative F0 Prediction: The current work on predicting F0 is preliminary and illustrative, not a full TTS system.
The authors envision scaling up their analyses to larger datasets (like audiobooks) and developing more complete prediction models for appropriate FO contours that are sensitive to synthesis context, moving towards more natural, discourse-aware TTS.
Summary updated