HUMIR at IEST-2018: Lexicon-Sensitive and Left-Right Context-Sensitive BiLSTM for Implicit Emotion Recognition

Behzad Naderalvojoud, Alaettin Ucan, Ebru Akcapinar SezerEMNLP 2018

TL;DR

This paper tackles the challenging task of implicit emotion recognition, where the emotion isn't explicitly stated but implied by the context around a target word. The authors propose a novel approach using Bidirectional Long Short-Term Memory (BiLSTM) networks that are sensitive to both the left and right context of the target word, combined with lexicon-based features. Their ensemble system achieved a strong performance, ranking sixth in the IEST-2018 shared task.

What Problem Does This Paper Address?

Recognizing emotions in text is a well-established area in Natural Language Processing (NLP). However, this paper focuses on a more nuanced problem: implicit emotion recognition. Unlike explicit emotion detection where the emotion is directly expressed (e.g., "I am happy"), implicit emotion recognition aims to identify the emotion conveyed by a target word that has been removed from its original context. This is akin to a word sense disambiguation task, where the target word can express different emotions depending on the surrounding text. The IEST-2018 shared task, which this paper participated in, specifically addresses this by asking participants to predict one of six categorical emotions (anger, disgust, fear, joy, sadness, surprise) for a masked target word in a tweet. This task is particularly challenging because the emotion is not directly tied to the target word itself but rather to the subtle cues within the tweet's context.

Methods

The core of the HUMIR system lies in its deep neural network architecture, primarily leveraging Bidirectional Long Short-Term Memory (BiLSTM) networks. BiLSTMs are effective for sequence modeling as they process information from both left-to-right and right-to-left, capturing dependencies across the entire input.

The authors propose several key methodological innovations:

  • Left-Right Context-Sensitive BiLSTM (LR-BiLSTM): This is a central contribution. Instead of processing the entire tweet as a single sequence, they split the context around the target word into left and right segments. Separate BiLSTM networks are applied to each segment, and their outputs are concatenated. This allows the model to specifically learn the semantic signature of the context immediately preceding and following the target word, which is crucial for inferring its implicit emotion.
  • Emotion-Specific Word Embeddings: To address the issue of out-of-vocabulary (OOV) words and to better capture emotional nuances, they fine-tune pre-trained word embeddings (TwitterGloVe) on their training data. They experiment with embeddings trained for different epochs, finding that embeddings from the first epoch yield the best results, suggesting that overfitting can occur with excessive fine-tuning.
  • Lexicon-Based and Emotion-Weight Features: In addition to context-sensitive features from BiLSTMs, they incorporate features derived from emotion lexicons (like AffectiveTweets) and a novel "emotion-weight" feature. This latter feature quantifies the relatedness of words in a tweet to each of the six emotion classes.
  • Ensemble Method (Bagging): To maximize performance, they combine multiple models using a Bagging ensemble. This involves training several models with different configurations and averaging their predictions to produce a more robust final output.

Datasets

The primary dataset used for training and evaluation is the IEST-2018 shared task dataset. This dataset consists of tweets with a target word masked, requiring the prediction of the implicit emotion. The authors also mention using the AffectiveTweets Weka package for lexicon-based features.

Key Insights and Ideas

  • Context is King (Especially Local Context): The success of the Left-Right Context-Sensitive BiLSTM highlights the critical importance of the immediate surrounding words for inferring implicit emotions. Treating left and right contexts separately and then combining their representations proved more effective than a single, global context analysis.
  • Fine-tuning Embeddings Matters, But Be Careful: While using pre-trained word embeddings is standard, fine-tuning them on the specific task domain (emotion recognition) can improve performance. However, the study demonstrates that over-tuning can lead to overfitting, emphasizing the need for careful hyperparameter selection.
  • Lexicons Still Have Value, But Not Always Enough: Lexicon-based features can complement deep learning models, but their effectiveness can be limited when the emotion is heavily dependent on syntactic structure or subtle contextual cues rather than explicit emotional words.
  • Anger and Surprise are Tricky: The confusion matrix reveals that anger and surprise are the most difficult emotions to distinguish, often being confused with each other or other emotions. This suggests that these emotions might share similar contextual expressions or that the available data has inherent ambiguities.

Key Results

The HUMIR system achieved a macro F-measure of 68.8 on the official test set, ranking sixth out of 30 participants in the IEST-2018 shared task.

Individually, the proposed Left-Right Context-Sensitive BiLSTM models (specifically LR-BiLSTM-3 and LR-BiLSTM-4) performed best among the individual models, achieving a macro F1-score of 67.8. The ensemble system, combining multiple models, further boosted this score to 68.6 (macro F1) and 68.8 (micro-average F1). The Lexicon-MLP model, relying solely on lexicon and emotion-weight features, showed significantly lower performance (49.81% accuracy on the development set), underscoring the importance of contextual information.

Limitations and Future Work

  • Ambiguity of Anger and Surprise: The paper acknowledges that anger and surprise emotions are frequently confused, indicating a limitation in distinguishing them. This is attributed to their similar contextual expressions and potential ambiguities in the data.
  • Limited Affective Clue Words: The authors suggest that a small number of affective clue words in tweets might contribute to the difficulty in emotion recognition.
  • Syntactic Structure's Influence: The paper notes that the syntactic structure of the context can significantly alter the expressed emotion, a factor that might not be fully captured by all models.

Future work could involve exploring more sophisticated methods for handling the ambiguity between anger and surprise, investigating how to better leverage syntactic information, and potentially developing new datasets with richer contextual cues for implicit emotion recognition.

Summary updated

BibTeX
@inproceedings{naderalvojoud2018humir,
  author = {Behzad Naderalvojoud and Alaettin Ucan and Ebru Akcapinar Sezer},
  title = {{HUMIR at IEST-2018: Lexicon-Sensitive and Left-Right Context-Sensitive BiLSTM for Implicit Emotion Recognition}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure