TL;DR
This paper explores whether incorporating Hindi word embeddings into an English aggression detection system can improve performance, especially in cases of code-switching. While their multi-lingual approach didn't significantly outperform a single-language model on the tested datasets, it highlights the challenges and nuances of cross-lingual representation learning for social media text.
What Problem Does This Paper Address?
Social media platforms are increasingly global, leading to a phenomenon known as "code-switching," where users mix languages within a single post. This poses a significant challenge for natural language processing (NLP) tasks like aggression detection. Traditional models often treat words from a different language as "out-of-vocabulary" (OOV) tokens, effectively ignoring potentially crucial information. This paper investigates whether leveraging pre-trained word embeddings from both English and Hindi, and aligning them into a common space, can help capture the nuances of code-switched text and improve aggression identification accuracy.
Methods
The authors developed a classification pipeline using a Bidirectional Gated Recurrent Unit (GRU) network, a type of recurrent neural network well-suited for sequence modeling. Their core innovation lies in how they handle word representations:
- Token Identification: Each token in the input text is identified as either an emoticon, a foreign (Hindi) token, or a standard English token.
- Emoticon Handling: Emoticons are represented by averaging the vectors of their Unicode descriptions.
- English Token Handling: For English words not found in the pre-trained embeddings, character trigrams are used to generate representations.
- Multi-Lingual Approach: The key experiment involves aligning pre-trained English and Hindi fastText word embeddings into a single vector space. This alignment is achieved using Singular Value Decomposition (SVD) to learn a linear transformation, aiming to make embeddings of similar words across languages (e.g., "fight" in English and "luchar" in Spanish, though Spanish wasn't used in this specific experiment) closer in the shared space.
- Single-Language Baseline: For comparison, a baseline system treats all Hindi words as OOV tokens, relying solely on English embeddings.
Datasets
The study utilizes data from the 2018 Trolling, Aggression and Cyberbullying shared task. The task provided datasets split by language (English and Hindi) and platform (Facebook and Twitter). For this research, the authors focused on the English Facebook (EN-FB) and English Twitter (EN-TW) datasets.
Key Insights and Ideas
- Code-Switching is Tricky: The paper directly confronts the challenge of code-switching, recognizing that ignoring foreign words can lead to missed signals in aggression detection.
- Alignment is Key for Multi-Lingual Embeddings: Simply combining embeddings from different languages isn't enough; they need to be aligned into a common space to be truly useful for cross-lingual understanding. The SVD-based alignment technique is a crucial component here.
- FastText's Advantage: The choice of fastText embeddings is motivated by their ability to handle OOV words by considering character n-grams, which is beneficial for dealing with misspellings and stylistic variations common in online text.
- Hypothesis Tested: The central hypothesis was that including aligned Hindi word embeddings would improve aggression classification accuracy on English data.
Key Results
The multi-lingual system did not yield a significant improvement over the single-language baseline on the validation data:
- EN-FB Dataset: Multi-lingual system achieved a weighted F1 score of 0.5315, while the single-language system scored 0.5355.
- EN-TW Dataset: Multi-lingual system achieved a weighted F1 score of 0.4389, while the single-language system scored 0.3477.
Interestingly, while the overall difference was not striking, the multi-lingual system performed better on the EN-TW dataset. The authors note that the confusion matrices reveal the classifier is better at identifying overt aggression and struggles more with distinguishing covert aggression from non-aggressive content. The system was submitted to the shared task and ranked 24th.
Limitations and Future Work
- Limited Hindi Token Frequency: The authors acknowledge that the number of Hindi tokens in the English dataset was too low to definitively prove their hypothesis. This suggests that code-switching might not have been a deeply characteristic feature of this specific dataset.
- SVD Alignment Quality: They also suggest that the SVD method for aligning embeddings might not have provided sufficiently good representations.
- Future Directions:
- Testing the hypothesis on datasets with more systematic code-switching.
- Exploring alternative architectures or combining classifiers to better distinguish between different types of aggression (e.g., aggressive vs. non-aggressive, and covert vs. overt aggression).
Summary updated