TL;DR
This paper introduces a novel framework, SKS, for hate speech detection that leverages sentiment knowledge sharing through a multi-task learning approach. By jointly training hate speech detection with sentiment analysis, SKS effectively captures nuanced sentiment and derogatory language, achieving state-of-the-art performance on public datasets.
What Problem Does This Paper Address?
The proliferation of hate speech on the internet poses a significant threat to society. While automatic detection of hate speech is crucial for intervention and mitigation, existing methods often struggle with the inherent complexity and subtlety of language. Rule-based and manual feature extraction approaches are brittle and require extensive domain knowledge. Traditional machine learning models, while better, can only capture shallow textual features. Deep learning models have shown promise by extracting latent semantic features, but they often overlook the crucial role of sentiment, especially external sentiment resources, which can significantly impact performance. This paper aims to bridge this gap by proposing a model that explicitly incorporates sentiment knowledge sharing to improve hate speech detection.
Methods
The proposed framework, SKS (Sentiment Knowledge Sharing), is built upon a multi-task learning architecture. The core idea is to leverage the strong correlation between hate speech detection and sentiment analysis.
- Input Layer: This layer enriches word embeddings by incorporating "category embeddings." A derogatory word dictionary is used to identify words that are likely to be part of hate speech. Each word in a sentence is then represented by a concatenation of its standard word embedding and its category embedding (indicating whether it's a derogatory word or not).
- Sentiment Knowledge Sharing Layer: This is the heart of the SKS model. It employs a multi-task learning framework to model the relationship between hate speech detection and sentiment analysis. Multiple feature extraction units are used, each consisting of a multi-head attention layer and a feedforward neural network. These units are designed to extract diverse features.
- Gated Attention Layer: A gated attention mechanism is used to dynamically select and fuse the outputs from the multiple feature extraction units. This allows the model to weigh the importance of different feature extractors based on the input, effectively adapting to task-specific needs.
- Pooling Layer: Maximum and average pooling are used to fuse the features extracted by the attention mechanism, creating a comprehensive sentence representation.
- Output Layer: A final feedforward neural network uses the fused features to predict whether a sentence constitutes hate speech.
Datasets
The paper evaluates the SKS model on two public hate speech datasets and one sentiment analysis dataset:
- SemEval-2019 Task 5 (SE): This dataset is specifically designed for multilingual hate speech detection. The authors use subtask A, focusing on hate speech detection.
- Davidson Dataset (DV): This dataset was collected from Twitter and is known for its imbalanced nature, with fewer hate speech examples.
- Sentiment Analysis (SA): A sentiment dataset from Kaggle 2018 is used to provide external sentiment knowledge for the multi-task learning component.
Key Insights and Ideas
- Sentiment is Key: The paper strongly emphasizes that sentiment, both from the target sentence and external resources, is a critical component for accurate hate speech detection. Simply relying on derogatory words or general semantic features is insufficient.
- Multi-Task Learning Synergy: Jointly training hate speech detection with sentiment analysis allows the model to learn shared representations and benefit from the correlation between the tasks, leading to improved performance and generalization.
- Gated Attention for Flexibility: The gated attention mechanism is crucial for effectively fusing features from multiple extraction units. It allows the model to adaptively select the most relevant features for each specific input, enhancing robustness.
- Derogatory Word Importance: While not the sole determinant, identifying derogatory words and their associated sentiment provides a strong signal for hate speech detection. The category embedding approach effectively incorporates this information.
Key Results
- State-of-the-Art Performance: The SKS model achieved the best performance in terms of F1-score on both the DV and SE datasets, outperforming several strong baselines, including SVM, LSTM, GRU, CNN-GRU, Universal Encoder, BERT, and GPT.
- Significant Improvement on SE: The SKS model showed a substantial improvement of nearly 10% in F1-score on the SE dataset compared to other models, highlighting its effectiveness in handling more nuanced and challenging hate speech instances.
- Ablation Study Findings: Ablation experiments confirmed the importance of sentiment knowledge sharing and category embedding, showing significant performance drops when these components were removed. The gated attention mechanism also contributed to performance gains.
Limitations and Future Work
- Dataset Scale Influence: The paper notes that the scale of the sentiment dataset relative to the hate speech dataset can impact performance. While a 2:1 ratio of sentiment to hate speech data showed peak performance, further optimization of data ratios is suggested.
- Nuance of Language: The authors acknowledge that the nuance of language can significantly affect hate speech detection performance, as seen in the differing results on the DV and SE datasets.
- Future Directions: The authors suggest future work could focus on better modeling and data selection for different types of hate speech, exploring the optimal type and scale of sentiment data, and further investigating the relationship between hate speech and sentiment analysis.
Summary updated