Acoustic-Prosodic Entrainment and Social Behavior

Rivka Levitan, Agustín Gravano, Laura Willson, Stefan Benus, Julia Hirschberg, Ani NenkovaNAACL 2012 · OpenReview

Abstract

Rivka Levitan, Agustín Gravano, Laura Willson, Štefan Beňuš, Julia Hirschberg, Ani Nenkova. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2012.

Introduction

In human conversation, people naturally and unconsciously adapt their behavior to match their partner—a phenomenon known as entrainment (or accommodation). While previous research has documented how we mimic vocabulary, syntax, and even posture, this paper investigates the raw acoustic and prosodic qualities of our voices, such as pitch, loudness, and speaking rate. The authors explore how this mimicry varies by gender and how it influences social perception and conversational efficiency.

Methodology

The researchers used the Columbia Games Corpus, which contains approximately 9 hours of spontaneous dialogue from cooperative computer games. They extracted eight key acoustic/prosodic features:

  • Intensity: Mean and maximum loudness.
  • Pitch: Mean and maximum frequency.
  • Speaking Rate: Syllables per second.
  • Voice Quality: Jitter (pitch variation), shimmer (amplitude variation), and noise-to-harmonics ratio (NHR).

To measure entrainment, they compared the similarity between conversational partners against a baseline of non-partners. These scores were then correlated with two types of data:

  1. Social Variables: Perceptions of behavior (e.g., "encouraging," "awkward") rated by crowdsourced workers.
  2. Objective Variables: Automated metrics of dialogue flow, such as turn latency (pauses) and interruptions.

Gender Dynamics in Entrainment

One of the study's primary findings is that entrainment is not a universal constant; it varies significantly based on the gender makeup of the pair. Mixed-gender (Female-Male) pairs were the most vocally adaptable, entraining on all eight features. Female-Female pairs followed with five features, while Male-Male pairs were the most rigid, entraining on only three.

Significant entrainment across acoustic features by gender pairing.

Social Perception vs. Dialogue Flow

The benefits of entrainment manifest differently depending on the pair. In mixed-gender interactions, entrainment is strongly tied to social perception. For example, listeners rated mixed-gender pairs who entrained on intensity as highly "encouraging."

Correlations between perceived social behaviors and acoustic entrainment across different gender pairings.

In contrast, for Male-Male pairs, entrainment is more crucial for the objective mechanics of the dialogue. Higher entrainment in these pairs was strongly correlated with reduced turn latency, meaning fewer awkward pauses and a tighter conversational flow.

The "Outlier" Effect

The study also conducted an "outlier" analysis to see how extreme vocal styles affect mimicry. Surprisingly, partners made a larger adjustment to match speakers with unusually loud or fast voices than they did for speakers with normal styles. This suggests that highly salient, noticeable vocal features trigger a stronger unconscious psychological urge to align.

Key Exceptions

While entrainment generally correlates with positive traits, "jitter" (roughness in the voice) proved to be an anomaly. Entraining on jitter was the only feature positively correlated with conversational awkwardness and negatively correlated with giving encouragement, suggesting that not all mimicry is socially beneficial.

Conclusion

Acoustic-prosodic entrainment is a powerful, unconscious tool that facilitates both social rapport and conversational efficiency. By understanding how these vocal adjustments vary by gender and context, researchers can better model human-computer interaction and social dynamics in dialogue.

Summary updated

BibTeX
@inproceedings{levitan2012acoustic,
  author = {Rivka Levitan and Agustín Gravano and Laura Willson and Stefan Benus and Julia Hirschberg and Ani Nenkova},
  title = {{Acoustic-Prosodic Entrainment and Social Behavior}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2012},
  url = {https://openreview.net/forum?id=HJWqXm-uWB},
}
Figure