PEMV: Improving Spatial Distribution for Emotion Recognition in Conversations Using Proximal Emotion Mean Vectors

Chen Lin, Fei Li, Donghong Ji, Chong TengNAACL 2025 · OpenReview

Abstract

Emotion Recognition in Conversation (ERC) aims to identify the emotions expressed in each utterance within a dialogue. Existing research primarily focuses on the analysis of contextual structure in dialogue and the interactions between different emotions. Nonetheless, ERC datasets often contain difficult-to-classify samples and suffer from imbalanced label distributions, which pose challenges to the spatial distribution of dialogue features. To tackle this issue, we propose a method that generates Proximal Emotion Mean Vectors (PEMV) based on emotion feature queues to optimize the spatial representation of text features. We design a Center Loss based on PEMVs to pull hard-to-classify samples closer to their respective category centers and employ Angle Loss to maximize the angular separation between different PEMVs. Furthermore, we utilize PEMV as a classifier to better adapt to the spatial structure of dialogue features. Extensive experiments on three widely used benchmark datasets demonstrate that our method achieves state-of-the-art performance and validates its effectiveness in optimizing feature space representations.

Introduction

Emotion Recognition in Conversation (ERC) is a critical task in natural language processing, yet it remains challenging due to the inherent ambiguity and contextual dependency of human speech. A single phrase can convey vastly different emotions depending on the speaker's history and the immediate conversational context. While modern models leverage Supervised Contrastive Learning (SupCon) to cluster similar emotions, they frequently struggle with two primary issues: hard-to-classify outliers and significant class imbalance (e.g., "Neutral" samples dominating rarer emotions like "Fear").

The Problem: Spatial Distribution Challenges

Current ERC models often fail to optimize the spatial distribution of dialogue features effectively. In the mathematical feature space, "hard" samples often reside far from their designated emotion clusters, leading to overlapping boundaries and misclassification.

Visualization of emotion feature space showing easy vs. hard-to-classify samples.

As shown in the visualization above, the proximity of different emotion clusters (like Joy and Neutral) and the presence of hard-to-classify samples (triangles) create messy decision boundaries that hinder model accuracy.

Methodology: The PEMV Framework

To address these spatial distribution issues, we introduce Proximal Emotion Mean Vectors (PEMV). This framework optimizes the feature space through a dynamic, queue-based approach.

The PEMV framework: generating mean vectors from feature queues to guide center and angle loss optimization.

1. Dynamic Feature Queues

Rather than calculating static global averages, the model maintains an Emotion Feature Queue for each category. These queues store the most recent features encountered during training, allowing the model to adapt to the evolving feature space during the learning process.

2. Generating PEMVs

By averaging the vectors within these queues, the model generates a Proximal Emotion Mean Vector. This vector serves as a moving target representing the ideal center for a specific emotion at a given point in the training process.

3. Dual-Loss Optimization

The framework employs two distinct loss functions to refine the spatial map:

  • Center Loss: Acts as a "distance penalty," pulling hard-to-classify samples (those exceeding a specific distance threshold) closer to their respective PEMV center.
  • Angle Loss: Maximizes the angular separation between different PEMVs (e.g., pushing the "Sad" center away from the "Mad" center), ensuring that clusters remain distinct and non-overlapping.

Finally, the PEMVs themselves are used as a classifier, where an utterance is assigned the emotion of the mean vector it most closely resembles.

Experimental Results

The PEMV method was evaluated against three major ERC benchmarks: MELD, EmoryNLP, and IEMOCAP. The results demonstrate that PEMV consistently achieves state-of-the-art (SOTA) performance.

  • MELD: Achieved a weighted-F1 of 67.95, outperforming previous SOTA models like SPCL+CL (67.25).
  • EmoryNLP: Reached 40.97, a significant jump over the previous best of 40.24.
  • IEMOCAP: Hit 70.65, slightly edging out the EACL model (70.41).

Qualitative analysis indicates that PEMV is particularly effective at distinguishing between "Neutral" and "Anger" samples, which are frequently confused with "Surprise" in baseline models.

Conclusion

By focusing on the spatial distribution of features through dynamic mean vectors and dual-loss optimization, PEMV provides a robust solution to the challenges of hard-to-classify samples and class imbalance in ERC. While currently limited to textual data, future research will explore integrating multimodal features—such as audio and visual cues—to further enhance the model's ability to capture the full spectrum of human emotional expression.

Summary updated

Abstract

Emotion Recognition in Conversation (ERC) aims to identify the emotions expressed in each utterance within a dialogue. Existing research primarily focuses on the analysis of contextual structure in dialogue and the interactions between different emotions. Nonetheless, ERC datasets often contain difficult-to-classify samples and suffer from imbalanced label distributions, which pose challenges to the spatial distribution of dialogue features. To tackle this issue, we propose a method that generates Proximal Emotion Mean Vectors (PEMV) based on emotion feature queues to optimize the spatial representation of text features. We design a Center Loss based on PEMVs to pull hard-to-classify samples closer to their respective category centers and employ Angle Loss to maximize the angular separation between different PEMVs. Furthermore, we utilize PEMV as a classifier to better adapt to the spatial structure of dialogue features. Extensive experiments on three widely used benchmark datasets demonstrate that our method achieves state-of-the-art performance and validates its effectiveness in optimizing feature space representations.
BibTeX
@inproceedings{lin2025pemv,
  author = {Chen Lin and Fei Li and Donghong Ji and Chong Teng},
  title = {{PEMV: Improving Spatial Distribution for Emotion Recognition in Conversations Using Proximal Emotion Mean Vectors}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=Ipss7KdRYu},
}
Figure