TL;DR
This paper tackles the challenge of identifying emotions and their underlying causes in conversations. They propose a novel two-step disjoint modeling approach, treating emotion classification and causal span extraction as separate but sequential tasks, inspired by question-answering techniques. This method achieved impressive results, securing 3rd place in the SemEval-2024 Task 3 competition.
What Problem Does This Paper Address?
Understanding human emotions in conversations is crucial for many applications, from improving human-computer interaction to analyzing social dynamics. However, simply identifying an emotion isn't enough; knowing why someone feels a certain way is often more insightful. This is particularly challenging in informal, multi-turn conversations where emotions can be subtly expressed and causes might span across multiple utterances. The SemEval-2024 Task 3, specifically Sub-Task #1, aimed to address this by requiring participants to identify utterance-level emotions and extract the specific text spans that caused those emotions, using only text data.
Methods
The authors propose a two-step disjoint modeling approach.
- Emotion Classification: They use a RoBERTa-base model, fine-tuned with a weighted cross-entropy loss. This weighting is crucial for handling imbalanced emotion distributions in the dataset. The input to this model includes the current utterance and the entire conversation context, aiming to capture how surrounding dialogue influences emotion expression.
- Causal Span Extraction: This step is framed as a Question Answering (QA) task. For utterances where an emotion (other than neutral) is predicted, the model is prompted with the utterance and its predicted emotion, and asked to identify the text span that caused it. They leverage a SpanBERT model, pre-trained on SQuAD 2.0 and then further fine-tuned on their specific task. This QA-like approach is novel for this problem.
During inference, the emotion classifier runs first to predict emotion labels, which then inform the causal span extractor.
Datasets
The primary dataset used for training and evaluation is the Multimodal Emotion-Cause Pair Extraction in Conversations (MC-ECPE) dataset, derived from the popular TV sitcom F.R.I.E.N.D.S. This dataset provides text, audio, and video, but for Sub-Task #1, only the text was utilized. The dataset consists of conversational utterances, with associated emotion labels and causal spans.
Key Insights and Ideas
- Context is King (for Emotion): The study found that contextual utterances before and after the target utterance play a significant role in accurately classifying emotions. This highlights the importance of understanding the broader conversational flow.
- QA for Cause Extraction Works: Framing causal span extraction as a QA problem proved effective. By asking "What caused this emotion?", the model can more directly pinpoint the relevant text segments.
- Disjoint Models are Competitive: The success of their two-step, disjoint approach suggests that separating the emotion classification and causal span extraction tasks can be a viable strategy, even outperforming some joint models.
- Class Imbalance Matters: Using weighted cross-entropy loss for emotion classification significantly improved performance, especially for under-represented emotion categories like disgust and fear.
Key Results
The proposed system achieved strong results in SemEval-2024 Task 3, Sub-Task #1:
- 3rd place overall based on the primary metric (weighted-average proportional F₁).
- 2nd place based on the auxiliary metric (weighted-average strict F₁), which emphasizes exact span matching.
- The model showed improvement over a text-only baseline from Wang et al. (2023).
- Ablation studies indicated that fine-tuning the causal span extractor with SQuAD 2.0 data significantly boosted performance.
Limitations and Future Work
The authors acknowledge several limitations:
- Future Utterance Causality: Their model struggled with cases where the cause of an emotion might appear in a future utterance, a phenomenon more likely in informal conversations.
- Neutral Utterances: While their model correctly skips neutral utterances for cause extraction, the unpredictability of span extraction when emotions are incorrectly predicted highlights the dependency on the emotion classifier.
- Resource Constraints: They opted for a simpler, full-context approach due to resource limitations, suggesting that more complex, jointly-trained, or multimodal models might further bridge performance gaps.
Future work could explore autoregressive context modeling or jointly trained models to better handle these complexities.
Summary updated