Acoustic features of anger utterances during natural dialog

Yoshiko Arimoto, Sumio Ohno, Hitoshi IidaInterspeech 2007

TL;DR

This paper investigates acoustic features that distinguish angry speech in natural Japanese dialogs, moving beyond acted emotions. They found that utterance duration, spectral tilt, and power variations are more indicative of anger than pitch, a finding that challenges some prior research. This work contributes to building more emotionally intelligent human-computer interaction systems.

What Problem Does This Paper Address?

Modern customer service systems are increasingly interactive, but they often lack the ability to understand the emotional state of the user. While automatic speech recognition (ASR) can process linguistic content, it misses crucial paralinguistic information like emotion. This gap limits the system's ability to respond appropriately, especially when users express negative emotions like anger. Previous research on emotion recognition often relied on acted speech, which may not reflect the nuances of spontaneous emotional outbursts during natural conversations. This paper aims to bridge this gap by identifying acoustic correlates of natural anger in dialogs, paving the way for more empathetic AI.

Methods

The researchers collected spontaneous anger utterances by simulating two types of pseudo-dialogs: a telephonic reservation system and a customer support contact center. To induce anger, operators either feigned misunderstanding or deliberately made errors. A crucial step was quantifying the "anger degree" of each utterance through a subjective evaluation by 12 university students, who rated utterances on a scale from 0 (not angry) to 5 (very angry).

Using this annotated dataset, they analyzed 11 acoustic parameters, categorized into pitch, power, duration/speaking rate, and voice-quality features. They employed multiple regression analysis with forward selection to identify which parameters best predict the subjective anger scores. This method systematically adds parameters to the model, selecting those that contribute most significantly to explaining the variance in anger degree.

Datasets

The study utilized a newly created dataset of spontaneous anger utterances collected from 10 university students (5 male, 5 female) participating in simulated Japanese dialogs. A total of 1400 utterances were selected for analysis, with 662 from male speakers and 738 from female speakers. This dataset was divided into training (2/3) and testing (1/3) sets for the estimation experiment.

Key Insights and Ideas

  • Natural Anger vs. Acted Emotion: The study highlights a critical distinction between emotions expressed in natural dialogs and those acted by professional speakers. This difference might explain why some traditional acoustic correlates of anger (like pitch) were less significant in their findings.
  • Duration and Power as Key Indicators: Surprisingly, utterance duration and features related to short-term power variations (like magnitude of power changes) emerged as strong predictors of anger. This suggests that how long someone speaks and how their voice intensity fluctuates are more telling than just their pitch.
  • Spectral Tilt Matters: A negative slope of the overall spectral tilt was also identified as a significant feature, indicating a change in the frequency distribution of the speech signal during anger.
  • Pitch Parameters Less Predictive: Contrary to some prior research, pitch parameters (like mean, min, max, and standard deviation of fundamental frequency) showed a weaker correlation with anger degree in this study. This could be due to the naturalistic nature of the data.

Key Results

The multiple regression analysis identified several key acoustic parameters contributing to anger estimation. The most significant parameters, in order of their contribution (indicated by Standard Partial Regression Coefficient - SPRC), were:

  • Dur (Duration): SPRC of 0.4933 (positive correlation, longer duration = more anger)
  • Clmean (Mean of first cepstral coefficient): SPRC of 0.1888 (negative correlation, suggests spectral tilt)
  • Rate (Speaking rate): SPRC of 0.1795 (positive correlation, faster rate = more anger)
  • Pmag (Magnitude of short-term power changes): SPRC of -0.1705 (negative correlation, smaller power changes = more anger)
  • Pmax (Short-term power max): SPRC of 0.1214 (positive correlation, higher max power = more anger)

The experiment achieved a standard error of 0.78 for the training set and 0.81 for the test set, indicating a reasonable ability to estimate the subjective anger scores.

Limitations and Future Work

The authors acknowledge that their findings might differ from previous studies due to the use of natural, spontaneous anger rather than acted emotions. They suggest that further research should investigate other pitch parameters not used in this study and explore parameters related to the temporal dynamics of dialog, such as speaker response time. Examining the influence of different qualities of emotional speech on acoustic features is also recommended.

Summary updated

BibTeX
@inproceedings{arimoto2007acoustic,
  author = {Yoshiko Arimoto and Sumio Ohno and Hitoshi Iida},
  title = {{Acoustic features of anger utterances during natural dialog}},
  booktitle = {Interspeech},
  year = {2007},
}
Figure