YNU-HPCC at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Using Multiple Prediction Headers

Hao Yang, Jin Wang, Xuejie ZhangACL 2025

TL;DR

This paper tackles the challenge of multilingual text-based emotion detection by proposing a novel approach using the RoBERTa model with modified prediction headers. Their method, which involves translating all text to English and then processing it, achieved a competitive score in the SemEval-2025 Task 11, demonstrating the effectiveness of this strategy for bridging language gaps in emotion analysis.

What Problem Does This Paper Address?

Multilingual sentiment and emotion analysis is a critical but challenging area in Natural Language Processing (NLP). While many advanced models like BERT and its successors excel at English, their multilingual capabilities are often limited, especially for low-resource languages or specific dialects. Directly training a single model on diverse languages can be inefficient due to parameter constraints and data scarcity. This paper addresses the "gap in text-based emotion" across languages, specifically for SemEval-2025 Task 11, which involves identifying six emotions (anger, disgust, fear, joy, sadness, surprise) in text from various languages. The core problem is how to effectively perform this task across a wide range of languages, including those with limited annotated data.

Methods

The YNU-HPCC team employed a strategy that leverages the strengths of powerful English-centric models. Their approach involved two main stages:

  1. Translation: All text data from the multilingual dataset was translated into English using Google Translate. This step aims to create a unified dataset that can be processed by models primarily trained on English.
  2. Emotion Classification: They utilized the RoBERTa (Robustly Optimized BERT Approach) model as their base. A key innovation was modifying the output head of the RoBERTa model. Instead of a single head predicting all six emotions simultaneously, they restructured it to predict each emotion independently. This means the model processes the input text and generates a probability for each of the six emotions separately.

To further enhance performance, they incorporated two techniques:

  • Focal Loss: This technique addresses class imbalance in the dataset by down-weighting the loss for well-classified examples and increasing focus on harder-to-classify, underrepresented emotions.
  • R-Drop (Regularized Dropout): This regularization technique stabilizes the training process by encouraging consistency across multiple forward passes of the same input, thereby improving generalization.

Datasets

The primary dataset used was from SemEval-2025 Task 11, Subtask A (Multi-label Emotion Detection). This dataset contains annotated training data with gold emotion labels for multiple languages. The paper mentions the dataset consists of approximately 60,000 instances, with significant class imbalances, particularly for "all-zero" (neutral) and "joy" labels.

Key Insights and Ideas

  • Translation as a Bridge: Translating all text to English proved to be an effective strategy for multilingual emotion detection, allowing them to leverage powerful English-based models like RoBERTa. This bypasses the complexities of training a truly multilingual model from scratch for this specific task.
  • Independent Emotion Prediction: Modifying the output head to predict each emotion independently, rather than all at once, significantly improved performance. This suggests that treating each emotion as a distinct classification problem can be more effective than a multi-label approach with a single output layer.
  • Focal Loss and R-Drop Synergy: The combination of Focal Loss and R-Drop was crucial for handling data imbalance and improving model stability, leading to better performance, especially for underrepresented emotions.

Key Results

The YNU-HPCC team achieved an official ranking score of 0.44 across all languages. Their experiments showed that:

  • The model with modified prediction headers (predicting each emotion independently) performed better than a single prediction header.
  • Training on the uniformly translated English dataset yielded better results than using the original multilingual dataset.
  • The combination of Focal Loss and R-Drop with the base DeBERTa model (though RoBERTa was their final choice, DeBERTa was also explored and discussed) showed significant improvements in accuracy, F₁-score, recall, and precision for most emotions.

Limitations and Future Work

  • Translation Quality: While Google Translate was effective, the paper acknowledges that machine translation performance can diverge from human expert translations, potentially impacting the accuracy of emotional expression and semantic similarity.
  • Low-Resource Languages: The paper notes that even with translation, performance on very low-resource languages might still be a challenge.
  • Future Work: The authors suggest that future research will focus on enhancing accuracy, particularly for underrepresented emotions, and exploring more advanced techniques to further bridge the gap in multilingual emotion detection. They also mention that their code is publicly available.

Summary updated

BibTeX
@inproceedings{yang2025ynu,
  author = {Hao Yang and Jin Wang and Xuejie Zhang},
  title = {{YNU-HPCC at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Using Multiple Prediction Headers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure