TL;DR
This paper tackles the challenging task of identifying offensive "spans" (specific phrases or words) within code-mixed Tamil-English social media comments. The authors propose a BiLSTM-CRF model, a standard deep learning architecture for sequence labeling, and achieve a modest F1 score of 0.1728, showing promise for shorter comments.
What Problem Does This Paper Address?
Identifying offensive language online is crucial for fostering healthier digital spaces and enabling automatic content moderation. However, this task becomes significantly more complex when dealing with code-mixed languages, where speakers seamlessly switch between two or more languages within a single utterance. This paper specifically addresses the challenge of pinpointing the exact segments (spans) of offensive content within code-mixed Tamil-English comments, a nuanced problem that goes beyond simple classification of an entire comment as offensive or not. Existing research has explored offensive content classification, but the precise identification of offensive spans in code-mixed languages, particularly Tamil, remains underexplored.
Methods
The authors employ a Bidirectional Long Short-Term Memory (BiLSTM) network coupled with a Conditional Random Field (CRF) layer, a well-established architecture for sequence labeling tasks. The process involves several key steps:
- Preprocessing: Comments are tokenized, and offensive spans are identified. The tokens are then converted into sequences and BIO (Beginning, Inside, Outside) encoded. This encoding scheme is standard for marking the start, continuation, and absence of a tagged entity.
- Encoding: Pre-trained GloVe embeddings, specifically trained on Twitter data, are used to represent words as 100-dimensional vectors. This approach leverages existing linguistic knowledge and is adapted for the code-mixed domain.
- BiLSTM-CRF Model: The core of the system is a BiLSTM layer that processes the input sequences in both forward and backward directions, capturing contextual information. The CRF layer then takes the BiLSTM outputs and predicts the most likely sequence of tags (BIO encoding), effectively identifying the offensive spans.
Datasets
The research utilizes a dataset released as part of a shared task on Toxic Span Identification. This dataset consists of:
- A training set with 4816 samples containing offensive span annotations.
- A testing set with 876 samples without annotations.
- A stripped-down version of the training set for validation and hyper-parameter tuning, containing span annotations for one or more words, but not necessarily the entire sentence.
Key Insights and Ideas
- Code-Mixing Adds Complexity: The paper highlights that code-mixing introduces a significant layer of complexity to offensive span identification, making it harder than analyzing single-language text.
- BiLSTM-CRF as a Baseline: While not novel in itself, the application of the BiLSTM-CRF architecture to this specific problem serves as a strong baseline for future research.
- Short Comments Show Promise: The model performs notably better on shorter comments (less than 30 characters), achieving a higher F1 score. This suggests that the model might be more adept at capturing local offensive cues in concise text.
Key Results
The developed BiLSTM-CRF system achieved an overall F1 score of 0.1728 on the test dataset. For comments with fewer than 30 characters, the system showed a significantly improved F1 score of 0.3890. This performance is noted as being lower than random baselines provided by the organizers for the overall task.
Limitations and Future Work
The authors acknowledge that their overall F1 score is significantly lower than random baselines, indicating room for improvement. They attribute this to the potential limitations of a single-layer BiLSTM in exploiting long-range contexts.
Future work suggested by the authors includes:
- Revisiting the problem with deeper architectures.
- Exploring different language models.
- Conducting a detailed study on the effect of various embedding types.
- Investigating the impact of increasing the number of layers and employing more advanced architectures.
Summary updated