Abstract
Researchers from KAIST and SNU have developed a framework to predict fine-grained, continuous emotion scores—Valence, Arousal, and Dominance (VAD)—using datasets that only contain coarse categorical labels. By mapping these labels into a 3D emotional space and training with Earth Mover’s Distance (EMD) loss, their model achieves state-of-the-art performance and excels in data-scarce scenarios, bridging the gap between abundant categorical data and rare dimensional annotations.
Introduction
In Affective Computing, emotions are typically modeled either categorically (discrete buckets like "Happy" or "Sad") or dimensionally (continuous VAD scores). While the VAD model is more descriptive and useful for applications like psychotherapy, VAD-annotated datasets are rare and expensive to produce. Conversely, categorical datasets are plentiful. This research addresses this "resource gap" by proposing a method to learn high-quality dimensional emotion detectors using only categorical supervision.
Methodology
The proposed pipeline utilizes a RoBERTa-Large backbone and introduces several key innovations to bridge the categorical-dimensional divide.
Mapping and Distributional Learning
Categorical labels (e.g., "joy") are mapped to specific coordinates in the 3D VAD space using the NRC-VAD lexicon. Instead of predicting a single point, the model predicts a probability distribution over the VAD dimensions. To train the model, categorical labels are sorted along each VAD dimension to create a target distribution for the model to approximate.
Earth Mover’s Distance (EMD) Loss
The core innovation is the use of Earth Mover’s Distance (EMD) loss. Unlike standard classification losses that penalize all incorrect predictions equally, EMD considers the metric distance between categories in the VAD space. For example, predicting "Joy" when the ground truth is "Ecstasy" results in a lower penalty than predicting "Sadness," as the former is closer in the emotional manifold.
Experimental Setup
The model was trained on categorical benchmarks including SemEval 2018 E-c (multi-label tweets), ISEAR (single-label events), and GoEmotions (Reddit comments). Evaluation was performed on EmoBank, a dataset containing 10k sentences with gold-standard VAD scores, to test generalization and regression performance.
Results
The framework demonstrates significant improvements over baseline models in both zero-shot and supervised settings.
Performance and State-of-the-Art
Even in a zero-shot scenario—without seeing any VAD-annotated data during training—the model achieved a 0.715 Pearson correlation for Valence on EmoBank. When fine-tuned, it outperformed previous state-of-the-art (SOTA) dimensional models across all three axes.
Key Insights
- Multi-label Advantage: Training on multi-label data (SemEval) yielded better results than single-label data (ISEAR), as multiple labels provide a denser "cloud" of points in the VAD space.
- Data Efficiency: The method serves as an effective pre-training step. In experiments using only 5-10% of training data, the proposed approach maintained high performance while standard regression baselines plummeted.
Discussion and Conclusion
While the model excels at predicting Valence, performance on Arousal and Dominance remains a challenge, likely because text conveys sentiment more clearly than physiological intensity. Future work may involve extending this framework to multimodal inputs (audio/video) to better capture these dimensions. Despite these limitations, the proposed EMD-based learning provides a robust way to leverage abundant categorical data for fine-grained emotional analysis.
Summary updated