SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation

Barbara Gendron, Gaël GuibonACL 2024

TL;DR

This paper introduces "SentEmoContext," a novel approach to emotion recognition in conversations that leverages metric learning and Siamese networks to effectively capture conversational context. By focusing on relative emotion similarities rather than absolute classifications, SentEmoContext achieves state-of-the-art performance on the DailyDialog dataset, particularly in macro F1 scores, while remaining computationally efficient.

What Problem Does This Paper Address?

Emotion Recognition in Conversation (ERC) is a challenging task due to the inherent subjectivity and ambiguity of human emotions, especially in text-based communication where non-verbal cues are absent. Existing methods often rely on complex graph-based representations or multimodal data, which can be computationally expensive and less efficient. Furthermore, ERC datasets are typically imbalanced, with a scarcity of emotional utterances compared to neutral ones, making it difficult for models to learn effectively. This paper aims to address these challenges by developing an efficient, context-aware model that can handle imbalanced data and generalize to different emotion granularities.

Methods

The core of SentEmoContext is a two-step metric learning training strategy using Siamese Networks.

  • Siamese Networks: These networks consist of multiple identical sub-networks that process different inputs (in this case, utterances or their contextual representations) and learn to map them into an embedding space. The goal is to learn a distance metric where similar emotions are closer together and dissimilar emotions are further apart.
  • Contextual Representations: To incorporate conversational context, the authors employ a BERT-like encoding approach. Utterances are first converted into sentence embeddings, and then a Transformer encoder layer with attention is used to capture the relationships between utterances within a dialogue. This creates a "contextual utterance representation."
  • Metric Learning with Triplet Loss: The model is trained using a triplet loss function. This loss operates on triplets of data samples: an anchor (a), a positive example (p) that is similar to the anchor, and a negative example (n) that is dissimilar. The loss aims to minimize the distance between the anchor and the positive while maximizing the distance between the anchor and the negative, with a margin to ensure separation.
  • Two-Step Training: The training process involves two phases:
  • Individual Emotion Prediction: A cross-entropy loss is used for direct emotion classification on utterance-level representations.
  • Relative Emotion Learning: A triplet loss is applied to the contextual utterance representations to learn the relative similarities between emotions. These two phases are trained iteratively, allowing each phase to strengthen the other.

  • Imbalance Handling: To address data imbalance, the authors use a weighted random sampler that assigns higher probabilities to under-represented emotion labels during batch creation. They also weight the cross-entropy loss based on inverse label frequencies.

Datasets

  • DailyDialog: This is the primary dataset used for training and evaluation. It contains over 10,000 dialogues with utterance-wise emotion labeling. The dataset includes six emotional labels (anger, disgust, fear, happiness, sadness, surprise) and a neutral label. For evaluation, the neutral label is excluded, focusing on the six emotional categories.

Key Insights and Ideas

  • Metric Learning for Context: The paper demonstrates that metric learning, by focusing on relative distances between emotions, is a powerful paradigm for ERC, especially when combined with contextual information. This approach is more robust to the inherent subjectivity of emotions than direct classification.
  • Efficiency through Siamese Networks and Sentence Transformers: The use of Siamese Networks and pre-trained sentence transformers (like MPNet, MiniLM, and RoBERTa) allows for a lightweight and efficient model that can be trained relatively quickly, even with limited epochs.
  • Contextual Representations are Crucial: The significant improvement in performance when incorporating conversational context highlights its importance for accurate emotion recognition. The Transformer encoder effectively captures these inter-utterance dependencies.
  • Imbalance Mitigation is Key: The paper emphasizes that simply using standard evaluation metrics that exclude the majority class is insufficient. Proactive strategies for handling data imbalance, like weighted sampling and loss weighting, are essential for achieving good performance across all emotion classes.
  • Generalizability: The metric learning approach, by learning relative relationships, holds promise for adapting to datasets with different emotion labels or granularities in the future.

Key Results

  • State-of-the-Art Macro F1: SentEmoContext achieves a macro F1 score of 57.71% on the DailyDialog dataset, outperforming previous state-of-the-art models like CD-ERC and TODKAT.
  • Satisfactory Micro F1: The model also achieves a micro F1 score of 57.75%, which is competitive.
  • Robustness: The model exhibits low variance (0.01) across metrics, indicating stability and reliability.
  • Efficiency: The model trains efficiently, requiring only about 20 minutes per epoch on a GPU, and converges within 5 epochs. It is also computationally lightweight compared to LLMs.
  • LLM Comparison: While LLMs show some emergent capabilities, they generally underperform SentEmoContext in this specific task, likely due to the challenges of prompt engineering and the need for extensive context incorporation.

Limitations and Future Work

  • LLM Limitations: The authors note that testing LLMs requires high-memory GPUs, and prompt tuning for specific tasks like ERC can be challenging. They also mention difficulties in controlling Falcon's output to generate a single label.
  • Context Size: For LLMs, the context window size can be a limitation, and adding extensive context to prompts can consume significant space.
  • Future Work:
    • Applying SentEmoContext to other datasets with added modalities.
  • Adapting the model to settings leaning towards meta-learning, leveraging its ability to learn relative label positions.
  • Exploring alternative triplet sampling strategies (e.g., batch-hard) for broader applicability.

Summary updated

BibTeX
@inproceedings{gendron2024sec,
  author = {Barbara Gendron and Gaël Guibon},
  title = {{SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure