TL;DR
This paper investigates how vocal characteristics like pitch, loudness, and voice quality signal dominance in group meetings. They found that dominant individuals tend to speak louder and with a "tense" voice, while less dominant individuals speak softer and with a "fearful" voice. This research is crucial for developing more expressive and socially aware speech synthesis systems.
What Problem Does This Paper Address?
The field of Social Signal Processing (SSP) aims to automatically understand social interactions through nonverbal cues. A key aspect of this is understanding "vocal social signals" – how we convey attitudes like dominance, agreement, or empathy not just through words, but through how we speak. While previous work has looked at dominance, this paper focuses specifically on the acoustic correlates of dominance in a realistic meeting scenario, aiming to extract patterns that can be used for expressive speech synthesis, rather than just classification. They want to bridge the gap between acoustic features and human perception of social signals.
Methods
The researchers employed a multi-step methodology:
- Acoustic Measures Extraction: They extracted a range of raw prosodic (e.g., fundamental frequency, energy, voicing rate) and spectral measures, as well as derived voice quality measures (both frame-based and utterance-based). These measures capture aspects like pitch, loudness, breathiness, and tension.
- Variability Reduction via PCA: To handle the inherent variability in natural speech, they used Principal Component Analysis (PCA). This technique helps identify the core underlying factors (principal components) that explain the most variance in the acoustic data, reducing redundancy and highlighting key predictors. They applied this to single-word utterances and then to longer meeting segments.
- Perceptual Interpretation: The loadings of the principal components were then interpreted in light of existing literature on the perceptual correlates of voice quality and social signals, particularly dominance. This step aims to translate the acoustic findings into human-understandable terms.
Datasets
The study utilized the AMI Meeting Corpus, specifically focusing on elicited meetings where participants played predefined roles. For the analysis of speaker roles, nine meeting sessions were used. For the dominance analysis, a subset of 11 sub-meetings from five sessions was selected, for which dominance annotations were available from three annotators.
Key Insights and Ideas
- Dominance is Acoustically Marked: The study confirms that dominance in meetings is not just perceived, but also acoustically encoded through prosody and voice quality.
- Loudness and Tension as Dominance Cues: The most dominant individuals tended to speak louder and with a more "tense" voice quality, while less dominant individuals spoke softer and with a "fearful" voice. This aligns with prior perceptual studies.
- Role-Dominance Correlation: The Project Manager (PM) role was strongly associated with dominance, while the Marketing Expert (ME) role was associated with lower dominance. This suggests that social roles within a meeting can influence vocal behavior.
- PCA for Feature Synthesis: The use of PCA is highlighted as a method to condense complex acoustic features into a more manageable, human-interpretable form, which is essential for developing rules for expressive speech synthesis.
Key Results
- Dominant Speakers: Tend to have higher energy, voicing rate, and fundamental frequency (Fp). Their voice quality measures (RCG, SKG, GOG, OQG) were generally lower, indicating a "tense" or "loud" vocal effort.
- Least Dominant Speakers: Showed the opposite tendencies, with lower energy, voicing rate, and Fp, and higher RCG, SKG, GOG, and OQG, suggesting a "soft" or "fearful" vocal effort.
- Speaker Roles: PCA analysis of speaker roles showed distinct clusters for the Project Manager (PM) and Marketing Expert (ME). PMs tended to exhibit loud vocal effort, while MEs leaned towards soft vocal effort.
- Dominance vs. Role: The study found that the PM role was acoustically correlated with dominance, and the ME role with lower dominance, supporting the idea that roles can predict dominance.
Limitations and Future Work
- Intra-speaker vs. Speaker-specific Effects: The authors acknowledge that the observed patterns might be influenced by individual speaking styles rather than solely by the social signal itself. Future work will investigate this by applying gradient normalization to more measures.
- Synthesis Focus: While the paper extracts correlates, the ultimate goal is synthesis. Future work will focus on transforming these acoustic findings into rules for generating expressive speech.
- Limited Data for Dominance: The dominance analysis was performed on a subset of the data with high annotator agreement, which, while ensuring reliability, might limit the generalizability of the findings.
Summary updated