The friendliness perception of dialogue speech

Jianhua Tao, Lixing Huang, Yongguo Kang, Jian Yu2006 · DOI

Abstract

This paper explores the acoustic and perceptual foundations of "friendliness" in dialogue. By utilizing a multi-dimensional "perception vector" that includes a novel "softness" metric, the researchers quantify the subjectivity of human listeners. A Classification and Regression Tree (CART) model was employed to identify key acoustic drivers, revealing that average pitch (F0 mean) and excitation strength (Ee) are the most significant factors in perceiving friendliness.

Introduction

In real-world interactions, such as customer service, the nuance of "friendliness" often outweighs binary emotional states. However, friendliness is inherently subjective and "fuzzy." This study addresses the lack of a formal framework to quantify this ambiguity and identifies the specific acoustic parameters—pitch, voice quality, and breathiness—that signal friendliness to human listeners.

Methodology: The Perception Vector

To capture the subjectivity of speech, the researchers introduced the Perception Vector. Instead of forcing listeners to choose a single emotion, this vector represents the probability across five basic emotions (anger, happiness, sadness, fear, and neutral) plus a "softness" degree.

Example of a perception vector including the softness metric.

A multi-tiered labeling system allowed listeners to provide "first" and "second" choices, which were weighted to calculate the vector. This approach preserves the nuance of human perception, where an utterance might be perceived as both "happy" and "neutral" simultaneously.

Acoustic Analysis and Modeling

The study utilized a corpus of 20 hours of real-world call center recordings. The researchers mapped 28 acoustic features—ranging from prosody (pitch and power) to voice quality (using the Liljencrants-Fant glottal flow model)—to perceived friendliness scores using a Classification and Regression Tree (CART) model.

Key Results and Insights

  • Softness is Critical: Friendliness is strongly and positively correlated with "softness"—a property independent of loudness—and "happiness." Conversely, a "very low friendliness" rating typically involves high anger combined with low softness and happiness.
  • Predictive Accuracy: The CART model correctly predicted 69% of friendliness categories using 28 leaf nodes.
  • Feature Importance:
    • F0 Mean (Average Pitch): Identified as the most influential factor in emotion perception.
  • Ee (Excitation Strength): The most critical parameter for modeling voice quality.
  • F0 Maximum Position: The most important "stress" feature for perceiving friendliness.

Conclusion

By moving beyond discrete emotion labels to a probabilistic perception vector, this research provides a more robust way to model social qualities in speech. These findings offer a roadmap for developing more natural and "friendly" speech synthesis systems by adjusting specific acoustic parameters like pitch and excitation strength.

Summary updated

Abstract

The paper is focused on the friendliness analysis and perception of dialogue speech.To do that, the paper uses a concept of the "perception vector" which contains the information of emotions and softness.In creating the "perception vector", and to simulate the perception ambiguity, the paper allows the listeners to label the speech with multiple emotions, and align them into "one choice", "first choice" and "second choice".Then, the paper makes the correlation analysis between friendliness and "perception vectors", the results disclose that the friendliness is positive correlation to "softness", "happiness" and "anger".Finally the paper traines a classification tree model to predict friendliness degree from acoustic features.With the classification tree model, we get the ranking scores of the acoustic parameters' importance for perceptually synthesized speech.Results shows that the F0 mean assumes the most important role in emotion perception, Ee is the most important parameter related to voice quality for the perception model.
BibTeX
@misc{tao2006friendliness,
  author = {Jianhua Tao and Lixing Huang and Yongguo Kang and Jian Yu},
  title = {{The friendliness perception of dialogue speech}},
  year = {2006},
  doi = {10.21437/speechprosody.2006-7},
}
Figure