SRL4E - Semantic Role Labeling for Emotions: A Unified Evaluation Framework

Cesare Campagnano, Simone Conia, Roberto NavigliACL 2022 · DOI

Introduction

Traditional emotion detection in Natural Language Processing (NLP) typically treats sentences as monolithic units, assigning a single label like "happy" or "sad." However, human expression is far more nuanced. A single sentence can convey multiple emotions, each with its own internal structure. The SRL4E framework addresses this by applying Semantic Role Labeling (SRL) to emotions, identifying not just what is felt, but who feels it, why, and at whom it is directed.

The Problem: Fragmentation in Emotion NLP

Before SRL4E, research into the semantic roles of emotions was fragmented. Various datasets existed, but they used different emotion taxonomies (such as Ekman vs. Plutchik) and inconsistent labeling formats. This fragmentation made it nearly impossible to compare models or develop systems that generalized across domains like news, social media, or literature.

Example of Semantic Role Labeling for Emotions (SRL4E) annotation.

The SRL4E Framework

The authors propose a unified schema that standardizes emotion analysis into three sub-tasks:

  1. Cue Identification: Locating the trigger word or phrase (e.g., "stand by").
  2. Emotion Classification: Mapping the cue to one of eight basic categories in Plutchik’s Wheel of Emotions.
  3. Role Identification: Extracting the spans for the Experiencer (the entity feeling the emotion), the Target (the entity the emotion is directed toward), and the Stimulus (the event or action causing the emotion).

Data Consolidation and Diversity

To create a robust benchmark, the researchers unified six heterogeneous datasets—ranging from 2012 US Election tweets to 19th-century novels—totaling nearly 32,000 instances. This process involved manual cleanup and mapping diverse labels to a common standard.

Distribution of emotion labels across the unified datasets.

As shown in the distribution above, different domains favor different emotions (e.g., "disgust" is prevalent in election tweets, while "joy" is common in blogs). By combining these, the model learns a more universal "grammar" of emotion.

Key Results and Insights

The core finding of the paper is the Power of Diversity. A model trained on the unified "All-in-one" dataset significantly outperformed models trained on single domains.

  • Generalization: The unified model achieved a 64.3% F1 score in emotion classification, whereas single-domain models often failed when tested on different text types.
  • Cue Anchoring: By focusing on the "Cue" as the anchor, the system can successfully handle complex sentences with multiple, overlapping emotions.
  • Cross-Lingual Transfer: The framework enabled effective transfer between English and Chinese, proving that the underlying structure of emotional expression transcends language barriers.

Conclusion and Future Directions

SRL4E provides a much-needed foundation for structured emotion analysis. While identifying complex "Stimulus" spans remains a challenge, the unified framework paves the way for more sophisticated, context-aware AI. Future work will likely focus on joint modeling to reduce error propagation between the identification and classification stages.

The code and unified datasets are available at https://github.com/SapienzaNLP/srl4e.

Summary updated

BibTeX
@inproceedings{campagnano2022srl,
  author = {Cesare Campagnano and Simone Conia and Roberto Navigli},
  title = {{SRL4E - Semantic Role Labeling for Emotions: A Unified Evaluation Framework}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.314},
}
Figure