UB_Tel-U at SemEval-2025 Task 11: Emotions Without Borders - A Unified Framework for Multilingual Classification Using Augmentation and Ensemble

Tirana Noor Fatyanosa, Putra Pandu Adikara, Rochmanu Purnomohadi Erfitra, Muhammad Rajendra Alkautsar Dikna, Sari Dewi Budiwati, CahyanaACL 2025

TL;DR

This paper introduces a unified framework for multilingual emotion detection that tackles the challenge of low-resource languages by consolidating data, employing augmentation, and using an ensemble of transformer models. The approach achieved competitive results, particularly in cross-lingual emotion detection, demonstrating the power of a holistic multilingual strategy.

What Problem Does This Paper Address?

The field of emotion recognition in text has largely focused on high-resource languages, leaving a significant performance gap for languages with limited annotated data. This disparity hinders the development of truly global emotion detection systems. SemEval-2025 Task 11 aimed to address this by providing a large multilingual dataset across three tracks: multi-label emotion detection, emotion intensity prediction, and cross-lingual emotion detection. The core challenge is to build robust models that can generalize across diverse languages and cultural nuances, especially when training data is scarce.

Methods

The UB_Tel-U system employs a multi-pronged strategy:

  • Unified Multilingual Dataset: Instead of creating separate models for each language, all data was merged into a single multilingual dataset. This allows the model to learn language-agnostic emotional representations.
  • Data Preprocessing: Key steps include lowercasing text, merging datasets, and standardizing emoticons and emojis into textual descriptions or emotion labels.
  • Data Augmentation: To enrich the training data and improve robustness, the team incorporated external datasets, specifically from SemEval 2018 Task 1, which included English, Spanish, and Arabic content. Both real and synthetic data augmentation were utilized.
  • Zero-Shot Classification: For cross-lingual tasks, a zero-shot classification approach using LLaMA 3.3 was implemented. This involved careful prompt engineering and a post-processing step to extract the desired emotion labels. The system also included a mechanism to translate non-English text to English, even transitively if direct translation was not possible.
  • Ensemble Learning: The final predictions were generated using ensemble methods. Two techniques were explored: majority voting and an "OR rule" ensemble. The OR rule was found to be more effective for multi-label classification, while majority voting performed better for emotion intensity.

Datasets

The primary dataset used was the multilingual dataset provided by SemEval 2025 Task 11. Additionally, the system was augmented with data from SemEval 2018 Task 1, which contained content in English, Spanish, and Arabic.

Key Insights and Ideas

  • Consolidation is Key: Merging data from all languages into a single multilingual dataset proved effective for learning language-agnostic emotional patterns, a crucial step for cross-lingual generalization.
  • Augmentation's Impact: Data augmentation, particularly using external corpora, significantly boosted performance across all tracks, demonstrating its value in enriching limited datasets.
  • Ensemble Power: Ensemble methods, especially the OR rule for multi-label tasks, were instrumental in achieving the best results by leveraging the complementary strengths of diverse models.
  • Transformer Dominance: The study confirmed the effectiveness of transformer-based models, with XLM-RoBERTa-base consistently outperforming others in multilingual settings. LLaMA 3.3 also showed strong zero-shot capabilities.
  • Low-Resource Challenges Persist: Despite the advancements, the system still struggled with certain low-resource languages (e.g., Emakhuwa, Yoruba), highlighting the ongoing difficulty of achieving robust performance without sufficient data.

Key Results

The UB_Tel-U system achieved competitive results:

  • Track A (Multi-label Emotion Detection): Ranked 11th for Afrikaans (afr) with a score of 0.5512.
  • Track B (Emotion Intensity Prediction): Ranked 9th for Ukrainian (ukr) with a Pearson correlation of 0.5365, and Russian (rus) achieved the highest score of 0.7817.
  • Track C (Cross-lingual Emotion Detection): Achieved 3rd place for Amharic (amh), Chinese (chn), Hindi (hin), Marathi (mar), Brazilian Portuguese (ptbr), Russian (rus), and Ukrainian (ukr).

The ensemble approach yielded the highest scores: 0.4997 in Track A, 0.5827 in Track B, and 0.4619 in Track C.

Limitations and Future Work

The authors acknowledge that the system struggled with some low-resource languages like Emakhuwa (vmw) and Yoruba (yor), indicating limitations in current techniques when faced with extremely scarce data. Emotion intensity prediction also remained challenging, showing considerable variance across languages.

Future work could explore more advanced data augmentation techniques such as back-translation, generative paraphrasing, and adversarial training. Additionally, incorporating domain-adaptive pretraining or language-specific adapters is suggested to further enhance model sensitivity to cultural and linguistic nuances. The authors also noted that due to time constraints, they were unable to submit their best-performing ensemble model.

Summary updated

BibTeX
@inproceedings{fatyanosa2025ub,
  author = {Tirana Noor Fatyanosa and Putra Pandu Adikara and Rochmanu Purnomohadi Erfitra and Muhammad Rajendra Alkautsar Dikna and Sari Dewi Budiwati and Cahyana},
  title = {{UB\_Tel-U at SemEval-2025 Task 11: Emotions Without Borders - A Unified Framework for Multilingual Classification Using Augmentation and Ensemble}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure