Relations between prominence and articulatory-prosodic cues in emotional speech

Jangwon Kim, Anil Ramakrishna, Sungbok Lee, Shrikanth Narayanan2016 · DOI

Abstract

This study investigates the relationship between perceived syllable prominence and various articulatory and prosodic cues in emotional speech. Utilizing electromagnetic articulography (EMA) and crowd-sourced prominence transcriptions, we evaluate the Converter/Distributor (C/D) model's traditional reliance on jaw excursion. Our findings indicate that the movement of the linguistically critical articulator (CA) and root-mean-square (RMS) energy are more robust predictors of prominence than jaw movement or fundamental frequency (f0), particularly across different emotional states. These results suggest that prominence is a complex, multi-modal phenomenon requiring more nuanced modeling than conventional methods provide.

Introduction

In linguistic theory, "prominence" refers to the quality by which certain syllables or words stand out relative to their neighbors. The Converter/Distributor (C/D) model has long served as a framework for understanding the physical realization of these prosodic features, typically using "jaw excursion" as a primary proxy for syllable magnitude. However, emotional speech introduces significant variability; for instance, "cold anger" may involve a clenched jaw despite high perceived prominence. This research aims to identify more reliable articulatory and acoustic correlates of prominence across five emotional states: neutral, angry, happy, fearful, and sad.

Methodology

The study employed a multi-modal data collection strategy involving two professional speakers (one male, one female) of American English.

  1. Articulatory Tracking: Electromagnetic Articulography (EMA) was used to track the 3D movements of sensors placed on the tongue tip, tongue dorsum, lower lip, and jaw.
  2. Perceptual Annotation: Rapid Prosody Transcription (RPT) was conducted via Amazon Mechanical Turk. Twenty naive listeners marked prominent words, which were then aggregated into a "p-score" (0 to 1) for each syllable.
  3. Feature Extraction: The researchers extracted 27 features, including acoustic metrics (f0, duration, RMS energy) and articulatory metrics (jaw movement and the movement of the "linguistically critical articulator" specific to each phoneme).

Results and Discussion

Emotional Impact on Prominence

Statistical analysis revealed that emotional state significantly influences perceived prominence. High-arousal emotions, such as anger and happiness, consistently yielded higher p-scores compared to low-arousal states like sadness or neutral speech.

Distribution of p-scores across five emotional categories (Neutral, Angry, Happy, Sad, Fearful).

As shown in the boxplot above, the median p-score for "Angry" and "Happy" speech is visibly higher than for "Neutral" or "Sad" speech, confirming that arousal is a primary driver of perceived prominence.

Articulatory and Prosodic Correlations

The study challenged the traditional C/D model by demonstrating that the Critical Articulator (CA) often outperforms jaw movement as a predictor of prominence. Furthermore, in the prosodic domain, RMS energy (loudness) proved to be a far more consistent cue than fundamental frequency (f0).

Correlation coefficients between p-scores and various articulatory/prosodic features across emotions and speakers.

The correlation table highlights that while f0 often shows weak correlations (frequently below 0.30), RMS energy and CA movements maintain high correlation coefficients (often exceeding 0.50 or 0.60), especially in high-arousal states like Anger for Speaker 2. This suggests that listeners rely more on intensity and specific articulatory effort than on pitch variations to identify prominent syllables in emotional contexts.

Conclusion

Our results indicate that the physical realization of prominence is a whole-mouth coordination effort rather than a simple jaw-drop mechanism. The high correlation of the Critical Articulator and RMS energy with p-scores suggests that future iterations of speech production models should incorporate a broader range of articulatory and acoustic behaviors.

Limitations of this study include the small speaker sample size and the use of linear correlation models. Future work will focus on larger datasets and non-linear statistical modeling to better capture the complex spatio-temporal relationships inherent in emotional prosody.

Summary updated

Abstract

This study investigates the relations between the degree of prominence and articulatory-prosodic cues in emotional speech.In particular, this study considers articulatory parameters driven from the Converter/Distributor (C/D) model.The goal is to obtain a better understanding of the link among syllable magnitude in the C/D model, the empirical way to measure it in literature, and syllable-level prominence, and to examine emotional variations appearing in this relation.Since prosodic variations are important cues for prominence and emotion in speech, relations with prosodic parameters (f0, energy, duration) are also considered.Electromagnetic articulography data of two speakers were used for analysis.The degree of prominence was computed on crowd-sourcing annotation data, using the Rapid Prosody Transcription.Results indicate that movements of linguistically critical articulator, energy, syllable magnitude measure are highly correlated with prominence; f0 is relatively less correlated.The movements of linguistically critical articulator tend to be more correlated than syllable magnitude measure.Inter-speaker variability and emotion-dependent variations are also reported.These results suggest complex relations between prominence and articulatory-prosodic cues.They also suggest that incorporating more articulatory and prosodic behaviors than the conventional way can better relate to perception of prominence.
BibTeX
@misc{kim2016relations,
  author = {Jangwon Kim and Anil Ramakrishna and Sungbok Lee and Shrikanth Narayanan},
  title = {{Relations between prominence and articulatory-prosodic cues in emotional speech}},
  year = {2016},
  doi = {10.21437/speechprosody.2016-183},
}
Figure