Dimensional Emotion Detection from Categorical Emotion

Sungjoon Park, Jiseon Kim, Seonghyeon Ye, Jaeyeol Jeon, Heeyoung Park, Alice OhEMNLP 2021 · OpenReview

Abstract

Researchers from KAIST and SNU have developed a framework to predict fine-grained, continuous emotion scores—Valence, Arousal, and Dominance (VAD)—using datasets that only contain coarse categorical labels. By mapping these labels into a 3D emotional space and training with Earth Mover’s Distance (EMD) loss, their model achieves state-of-the-art performance and excels in data-scarce scenarios, bridging the gap between abundant categorical data and rare dimensional annotations.

Introduction

In Affective Computing, emotions are typically modeled either categorically (discrete buckets like "Happy" or "Sad") or dimensionally (continuous VAD scores). While the VAD model is more descriptive and useful for applications like psychotherapy, VAD-annotated datasets are rare and expensive to produce. Conversely, categorical datasets are plentiful. This research addresses this "resource gap" by proposing a method to learn high-quality dimensional emotion detectors using only categorical supervision.

Methodology

The proposed pipeline utilizes a RoBERTa-Large backbone and introduces several key innovations to bridge the categorical-dimensional divide.

Mapping and Distributional Learning

Categorical labels (e.g., "joy") are mapped to specific coordinates in the 3D VAD space using the NRC-VAD lexicon. Instead of predicting a single point, the model predicts a probability distribution over the VAD dimensions. To train the model, categorical labels are sorted along each VAD dimension to create a target distribution for the model to approximate.

Overview of the model architecture and training pipeline using EMD loss to map categorical labels to VAD space.

Earth Mover’s Distance (EMD) Loss

The core innovation is the use of Earth Mover’s Distance (EMD) loss. Unlike standard classification losses that penalize all incorrect predictions equally, EMD considers the metric distance between categories in the VAD space. For example, predicting "Joy" when the ground truth is "Ecstasy" results in a lower penalty than predicting "Sadness," as the former is closer in the emotional manifold.

Experimental Setup

The model was trained on categorical benchmarks including SemEval 2018 E-c (multi-label tweets), ISEAR (single-label events), and GoEmotions (Reddit comments). Evaluation was performed on EmoBank, a dataset containing 10k sentences with gold-standard VAD scores, to test generalization and regression performance.

Results

The framework demonstrates significant improvements over baseline models in both zero-shot and supervised settings.

Performance and State-of-the-Art

Even in a zero-shot scenario—without seeing any VAD-annotated data during training—the model achieved a 0.715 Pearson correlation for Valence on EmoBank. When fine-tuned, it outperformed previous state-of-the-art (SOTA) dimensional models across all three axes.

Comparison of the proposed model against existing baselines on the EmoBank regression task.

Key Insights

  • Multi-label Advantage: Training on multi-label data (SemEval) yielded better results than single-label data (ISEAR), as multiple labels provide a denser "cloud" of points in the VAD space.
  • Data Efficiency: The method serves as an effective pre-training step. In experiments using only 5-10% of training data, the proposed approach maintained high performance while standard regression baselines plummeted.

Discussion and Conclusion

While the model excels at predicting Valence, performance on Arousal and Dominance remains a challenge, likely because text conveys sentiment more clearly than physiological intensity. Future work may involve extending this framework to multimodal inputs (audio/video) to better capture these dimensions. Despite these limitations, the proposed EMD-based learning provides a robust way to leverage abundant categorical data for fine-grained emotional analysis.

Summary updated

BibTeX
@inproceedings{park2021dimensional,
  author = {Sungjoon Park and Jiseon Kim and Seonghyeon Ye and Jaeyeol Jeon and Heeyoung Park and Alice Oh},
  title = {{Dimensional Emotion Detection from Categorical Emotion}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=jD3te-MYyR2},
}
Figure