How Interlocutors Coordinate with each other within Emotional Segments?

Firoj Alam, Shammur Absar Chowdhury, Morena Danieli, Giuseppe RiccardiCOLING 2016

TL;DR

This paper investigates how speakers coordinate their behavior within emotional segments of conversations, specifically in call centers. They found that interlocutors exhibit the most coordination during anger segments, suggesting that emotional states significantly influence conversational dynamics.

What Problem Does This Paper Address?

Understanding how people coordinate their behavior during conversations is crucial for building more natural and effective human-computer interaction systems. While previous research has explored emotion recognition and turn-taking dynamics separately, this paper tackles the challenge of understanding how these two aspects intertwine within specific emotional states. Specifically, they aim to uncover how interlocutors adjust their communication styles and behaviors in response to each other's emotions, focusing on the coordination within segments characterized by anger, frustration, and empathy. This is particularly relevant for applications like customer service, where managing emotional interactions is key to user satisfaction.

Methods

The researchers employed a multi-faceted approach to analyze conversational coordination. Their methodology involved:

  • Corpus Creation and Annotation: They utilized a dataset of Italian human-human call center conversations. Crucially, these conversations were manually annotated for emotional states (agent's empathy, customer's anger and frustration) and overlap discourse (competitive vs. non-competitive).
  • Feature Extraction: They extracted three main types of features:
  • Lexical Features: Using bag-of-words models and TF-IDF, they captured the word usage within emotional segments.
  • Psycholinguistic Features: They leveraged the Linguistic Inquiry and Word Count (LIWC) tool to analyze features related to psychological processes, personal concerns, and linguistic styles.
  • Turn-Taking Features: They developed a system to extract features like participation equality, percentage of overlaps, and duration of different turn types (e.g., competitive vs. non-competitive overlaps, pauses, lapses).
  • Analysis Techniques:
    • Cosine Similarity: Used to measure lexical similarity between agent and customer segments.
  • Pearson Correlation: Applied to assess the correlation between psycholinguistic features of interlocutors.
  • Multilevel Logistic Regression: Employed to understand the association between turn-taking features and emotional segments.

Datasets

The study used a custom-annotated corpus of Italian human-human spoken conversations from call centers providing after-sales customer care. This corpus comprised 1894 customer-agent conversations, totaling over 210 hours. A subset of 565 conversations (approximately 62 hours) was used for overlap discourse annotation. The corpus was annotated for:

  • Emotional States: Agent's empathy (Emp), Customer's anger (Ang), Customer's frustration (Fru). Preceding segments were also labeled (Pre.Ang, Pre.Fru, Pre.Emp). Non-emotional segments (NoEmo) were also included.
  • Overlap Discourse: Competitive (Cmp) and Non-Competitive (Ncm) overlaps.

Key Insights and Ideas

  • Anger as a Coordination Hotspot: The most striking finding is that interlocutors exhibit significantly higher coordination (both lexically and psycholinguistically) during anger segments compared to other emotional states. This suggests that anger, perhaps due to its intensity or the need for resolution, triggers a stronger drive for mutual adjustment.
  • Empathy's Unique Dynamics: While empathy segments showed less overall coordination, they were characterized by an increase in non-competitive overlaps. This implies a different mode of interaction, possibly where the agent is actively trying to understand and support the customer without necessarily mirroring their emotional state.
  • Preceding Context Matters: The study highlights the importance of the preceding emotional context. For instance, cues of anger can be detected in preceding segments, and the length of pauses changes significantly before anger versus frustration.
  • Turn-Taking as a Signal: Features like the percentage of overlaps and the duration of non-competitive overlaps positively influence empathic segments, suggesting these are key indicators of coordinated supportive interaction.

Key Results

  • Lexical Similarity: Interlocutors entrain each other more in non-overlapping turns during customer anger segments (similarity = 0.181, p < 0.05). Competitive overlaps showed the highest similarity in preceding-anger and anger segments (0.035 and 0.031 respectively).
  • Psycholinguistic Correlation: Non-overlapping turns in anger segments showed higher correlation values for psychological features (anxiety, affect, sad) compared to other segments.
  • Turn-Taking Regression: Non-competitive overlaps (MedianNcm) had a significant positive effect on empathic segments. For anger segments, the length of non-competitive overlaps (MedianNcm) had a positive association, while the length of lapses (MedianLb) had a negative effect.
  • Overlap Duration: Competitive overlap duration was significantly higher in anger and frustration segments compared to empathy and no-emotion segments.

Limitations and Future Work

  • Preliminary Study: The authors acknowledge this is a preliminary study, and they plan to further investigate these feature sets for classifying emotional states and turn-taking discourse.
  • Domain Specificity: The study is focused on call center conversations. The generalizability of these findings to other conversational domains might be limited.
  • Manual Annotation: While necessary for ground truth, manual annotation is time-consuming and can be subjective. Future work could explore more automated annotation methods.
  • Complexity of Emotions: The study focused on a few key emotions. Future research could explore a wider range of emotional states and their impact on coordination.

Summary updated

BibTeX
@inproceedings{alam2016how,
  author = {Firoj Alam and Shammur Absar Chowdhury and Morena Danieli and Giuseppe Riccardi},
  title = {{How Interlocutors Coordinate with each other within Emotional Segments?}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
}
Figure