EmotionIC: Emotional Inertia and Contagion-driven Dependency Modelling for Emotion Recognition in Conversation

Yingjian Liu, Jiang Li, Xiaoping Wang, Zhigang Zeng

TL;DR

This paper introduces EmotionIC, a novel approach for emotion recognition in conversations that explicitly models emotional inertia (a speaker's tendency to maintain an emotion) and emotional contagion (how one person's emotion influences another's). By integrating identity-aware attention and a novel skip-chain conditional random field, EmotionIC achieves state-of-the-art performance on several benchmark datasets.

What Problem Does This Paper Address?

Recognizing emotions in conversations (ERC) is crucial for applications like empathetic dialogue systems and social media analysis. However, existing methods often struggle to capture the dynamic nature of emotions in dialogues. They tend to either focus too narrowly on recent utterances (limiting context) or overemphasize global relevance (losing temporal information). Crucially, they often overlook the psychological phenomena of emotional inertia and contagion, which significantly influence how emotions unfold and spread between participants in a conversation. This paper aims to bridge this gap by developing a model that explicitly accounts for these inter- and intra-speaker emotional dynamics.

Methods

EmotionIC is built upon three core components:

  • Identity Masked Multi-Head Attention (IMMHA): This module captures global contextual dependencies by leveraging multi-head attention. It's "identity-masked" to differentiate between a speaker's own past emotions (self-dependency) and the emotions of other participants (other-dependency). This helps in understanding how a speaker's identity influences their emotional state and how they perceive others' emotions.
  • Dialogue-based Gated Recurrent Unit (DiaGRU): This component focuses on local, temporal dependencies. It refines the contextual information by considering the immediate past utterances and the speaker's identity. DiaGRU uses "reset gates" that are sensitive to emotional inertia (when the self-dependency gate is high) and emotional contagion (when the other-dependency gate is high), allowing it to dynamically weigh these influences.
  • Skip-chain Conditional Random Field (SkipCRF): This is the classification layer designed to model complex emotional flows. Unlike traditional linear-chain CRFs that only consider immediate neighbors, SkipCRF introduces "skip connections" to capture higher-order dependencies. This allows it to model emotional interactions across non-adjacent utterances, effectively simulating emotional propagation and inertia over longer conversational spans.

Datasets

The proposed EmotionIC model was evaluated on four widely used benchmark datasets for emotion recognition in conversation:

  • IEMOCAP: Interactive Emotional Dyadic Motion Capture database.
  • DailyDialog: A manually labeled multi-turn dialogue dataset.
  • MELD: A multimodal multi-party dataset for emotion recognition in conversations.
  • EmoryNLP: A dataset for emotion detection in textual conversations.

The paper notes that for DailyDialog, they exclude the "Neutral" class due to its extreme majority to better evaluate performance on other emotions.

Key Insights and Ideas

  • Emotional Inertia and Contagion are Key: The paper strongly argues that explicitly modeling these psychological phenomena is crucial for accurate ERC, moving beyond simple contextual analysis.
  • Identity Matters: Differentiating between self-dependency and other-dependency, and how these relate to speaker identity, is vital for understanding emotional dynamics.
  • Higher-Order Dependencies are Necessary: Simple linear-chain models are insufficient for capturing the complex, non-local emotional flows in conversations. SkipCRF's ability to look beyond immediate neighbors is a significant advancement.
  • Feature Extraction and Classification Synergy: EmotionIC's success stems from effectively integrating advanced feature extraction (IMMHA, DiaGRU) with a sophisticated classification mechanism (SkipCRF) that models the sequential nature of emotions.

Key Results

EmotionIC achieved state-of-the-art performance across the four benchmark datasets. For instance, on the IEMOCAP dataset, it achieved a weighted F1 score of 69.61 and an accuracy of 69.44, outperforming existing methods. The paper highlights that EmotionIC demonstrates strong improvements, particularly on longer conversations like IEMOCAP, where it effectively captures long-distance dependencies. Ablation studies confirmed the effectiveness of each proposed module (IMMHA, DiaGRU, and SkipCRF), showing performance degradation when any were removed or replaced with simpler alternatives.

Limitations and Future Work

The authors acknowledge several limitations:

  • Short Conversation Length: Datasets with very short conversations (like MELD) pose challenges for contextual modeling.
  • Class Imbalance: The prevalence of the "Neutral" emotion in datasets like DailyDialog can bias models.
  • Similar Emotions: Distinguishing between very similar emotions (e.g., Angry vs. Frustrated) remains difficult.
  • Commonsense Knowledge: The model's performance on datasets like EmoryNLP, which require more commonsense reasoning, is somewhat limited.

Future work will focus on addressing these challenges by exploring the use of commonsense knowledge and multimodal methods to better capture emotional mutations. They also aim to improve the modeling of emotional contagion, which they found to be more challenging than emotional inertia.

Summary updated

BibTeX
@article{liu2024emotionic,
  author = {Yingjian Liu and Jiang Li and Xiaoping Wang and Zhigang Zeng},
  title = {{EmotionIC: Emotional Inertia and Contagion-driven Dependency Modelling for Emotion Recognition in Conversation}},
  journal = {Science China Information Sciences},
  year = {2024},
  doi = {10.1007/s11432-023-3908-6},
}
Figure