TL;DR
This paper presents a deep learning approach for classifying emotions in tweets where emotion-bearing words have been removed. By combining BiLSTM and LSTM models with attention mechanisms and employing a soft voting ensemble strategy, the authors achieved a significant improvement in performance, securing 7th place in the WASSA2018 Implicit Emotion Shared Task.
What Problem Does This Paper Address?
Sentiment analysis, a well-established field, has seen a surge in interest due to the proliferation of social media platforms like Twitter. However, a specific challenge arises when analyzing tweets where the explicit emotion-carrying words are missing, making it difficult to infer the intended emotion. This paper tackles the WASSA2018 Implicit Emotion Shared Task, which aims to classify the emotion of these "excluded words" into six categories (sad, joy, disgust, surprise, anger, fear) based on the surrounding tweet context. This task is particularly challenging because the crucial emotional cues are deliberately omitted.
Methods
The core of the proposed method lies in leveraging deep learning architectures with attention mechanisms. The authors explored two primary models:
- BiLSTM-Attention: This model uses a Bidirectional Long Short-Term Memory (BiLSTM) network to process the tweet sequence, allowing it to capture context from both past and future words. An attention mechanism is then applied to weigh the importance of different words in the tweet for emotion representation.
- LSTM-Attention: Similar to the BiLSTM-Attention model, but it utilizes a standard Long Short-Term Memory (LSTM) network.
Both models incorporate an embedding layer (using pre-trained Word2vec-twitter embeddings), the respective RNN layer, an attention layer, and finally, two dense layers with a softmax activation for classification. A key aspect of their approach is the experimentation with different dropout rates to mitigate overfitting.
To further enhance performance, the authors employed a soft voting ensemble strategy. This involves combining the predictions of multiple models (specifically, the BiLSTM-Attention and LSTM-Attention models with various dropout rates) by taking a weighted average of their predicted probabilities. The class with the highest aggregated probability is then chosen as the final prediction.
Datasets
The experiments were conducted using datasets provided by the WASSA2018 Implicit Emotion Shared Task. These datasets consist of:
- Training set: 153,383 tweets
- Validation set: 9,591 tweets
- Test set: 28,757 tweets
The tweets were incomplete, with emotion words removed and marked as [#TRIGGERWORD#].
Key Insights and Ideas
- Attention is crucial for implicit emotion: The attention mechanism proved vital in allowing the models to focus on the most relevant parts of the tweet context to infer the emotion of the missing word.
- BiLSTM offers a slight edge over LSTM: The BiLSTM-Attention model generally performed slightly better than the LSTM-Attention model, likely due to its ability to capture both forward and backward context more effectively.
- Ensembling significantly boosts performance: The soft voting ensemble strategy was instrumental in achieving the final strong results, demonstrating the power of combining multiple models.
- Dropout tuning is important: Experimenting with different dropout rates helped in finding optimal configurations for the individual models and contributed to the overall robustness.
Key Results
The proposed method achieved a macro F1 score of 0.685, securing 7th position out of 30 teams in the WASSA2018 Implicit Emotion Shared Task. This represents a 12.5% improvement over the baseline method.
- The ensemble approach outperformed individual models, with the final ensemble achieving a macro F1 score of 0.685.
- Among the individual models, BiLSTM-Attention models generally showed slightly better performance than LSTM-Attention models across various dropout rates.
- The system performed best on the "joy" emotion class, achieving high precision and recall, while the "anger" class had the lowest F1 score.
Limitations and Future Work
The authors acknowledge several avenues for future research:
- Exploring other ensemble methods: Investigating hard voting and stacking techniques to see if they yield better results than soft voting.
- Combining word and character embeddings: Experimenting with different weights for word and character embeddings, and potentially incorporating new embedding algorithms like ELMo.
- Leveraging more textual features: Exploring the use of emotion icons and other textual cues to further improve performance.
Summary updated