Acoustic-prosodic analysis of attitudinal expressions in German

Hansjörg Mixdorff, Angelika Hönemann, Albert RilliardInterspeech 2015

TL;DR

This paper analyzes the acoustic-prosodic features of German utterances expressing 16 different social attitudes. By applying the Fujisaki model and examining features like pitch, intensity, and voice quality, the researchers identify patterns that distinguish these attitudes, offering insights into cross-cultural communication and the acoustic correlates of human emotion.

What Problem Does This Paper Address?

Understanding and accurately conveying social attitudes is crucial for effective human communication. While linguistic content carries meaning, the way we say something – our prosody – conveys a wealth of information about our emotional state, social intentions, and relationship with our interlocutor. However, the acoustic realization of these attitudinal expressions can vary significantly across languages and cultures, leading to potential misunderstandings. Previous research has often relied on simplified emotional categories or acted performances. This paper aims to address this by conducting a detailed acoustic-prosodic analysis of a wider range of social attitudes in German, using a framework that considers the social context and interactional roles, and by eliciting these attitudes from native speakers in a more naturalistic setting.

Methods

The researchers employed the Fujisaki model for prosodic analysis, which decomposes the fundamental frequency (F0) contour into components representing phrase and accent commands. This model was applied to analyze F0, speech rate, and intensity. Additionally, voice quality features such as harmonics-to-noise ratio (HNR), jitter, and shimmer were extracted for vowels within the target utterances. The data was force-aligned at the phone level and manually checked. Statistical analysis, specifically Mann-Whitney-U tests, was used to determine significant differences in prosodic features between attitudes.

Datasets

The study utilized a newly elicited dataset of German utterances. Twenty native German speakers (11 female, 9 male) with academic backgrounds produced short target sentences ("Eine Banane" and "Marie tanzte") in the context of short dialogues designed to elicit 16 specific social attitudes (e.g., arrogance, politeness, surprise, irritation). Each speaker produced two instances of each attitude. For the analysis, one instance was selected based on prior perception ratings or by the researchers themselves if ratings were unavailable.

Key Insights and Ideas

  • Attitudes have distinct acoustic footprints: While subtle, different social attitudes are associated with measurable differences in prosodic features, particularly in F0 range and voice quality.
  • The Fujisaki model is useful for attitudinal analysis: The model's ability to break down F0 contours into phrase and accent commands proved valuable in identifying differences related to attitudes.
  • Voice quality plays a role: Features like HNR, jitter, and shimmer, which relate to the breathiness or roughness of the voice, show significant variation across attitudes and can be indicative of underlying emotional states or communicative strategies.
  • Cultural nuances exist: While some cross-cultural similarities in prosodic expression of affect are expected (e.g., the "frequency code" relating pitch to dominance), the study acknowledges that culture-specific uses of prosodic parameters are also important for mutual understanding.
  • Clustering of attitudes: The analysis revealed groupings of attitudes based on their acoustic profiles, suggesting that certain attitudes share similar prosodic realization strategies. For instance, negative attitudes like arrogance and contempt clustered together, as did positive attitudes like admiration and politeness.

Key Results

  • Accent Command Amplitude (Aa): Attitudes like QUES (question), DOUB (doubt), and SURP (surprise) were characterized by significantly larger F0 ranges (higher Aa) compared to attitudes like CONT (contempt) and WOEG (walking-on-eggs).
  • Phone Duration: Neutral statements (DECL) and questions (QUES) were produced at the highest phone rate (shortest durations), while DOUB and IRRI (irritation) were produced most slowly.
  • Harmonics-to-Noise Ratio (HNR): Attitudes like SURP, ADMI, DOUB, and IRON showed lower HNR values, potentially indicating more breathy or less clear vocalizations, while QUES and DECL had higher HNR.
  • Jitter and Shimmer: While differences were generally small, some trends emerged. For instance, SURP and SEDU (seductiveness) showed higher jitter, potentially linked to excitement or nervousness. IRRI and AUTH (authority) exhibited lower jitter, suggesting a more controlled voice. Local shimmer differences were less pronounced but showed some inverse correlation with HNR.
  • Significant Differences: A matrix of significant differences (p < 0.01) between attitudes based on various acoustic features was generated, highlighting which features are most effective in distinguishing specific attitudinal pairs.

Limitations and Future Work

The authors acknowledge the small size of their sample, with only two short utterances per attitude per speaker. They also note that the perceived convincingness of the elicited attitudes varied, and they opted not to discard data to preserve the breadth of their analysis. Future work will explore the contribution of the visual channel (facial expressions) to attitudinal expression, as previous research suggests it plays a significant role. Further investigation will also focus on how well intended attitudes are identified by native listeners and whether the acoustically identified clusters of attitudes are confirmed by perception studies. The authors also point out that their metric-based analysis may miss other acoustic cues, such as lip smacks or smiles.

Summary updated

BibTeX
@inproceedings{mixdorff2015acoustic,
  author = {Hansjörg Mixdorff and Angelika Hönemann and Albert Rilliard},
  title = {{Acoustic-prosodic analysis of attitudinal expressions in German}},
  booktitle = {Interspeech},
  year = {2015},
}
Figure