TL;DR
This paper investigates how incorporating user context (like follower networks and user vocabulary) impacts hate speech detection models. By using explainability techniques, they show that user features significantly improve performance and alter the model's learned feature space, but also introduce new forms of bias.
What Problem Does This Paper Address?
Online hate speech is a pervasive problem, and automatically detecting it is crucial due to the sheer volume of content. While machine learning models have improved, they still struggle with implicit hate speech and sarcasm. Previous work has shown that adding "user features" (information beyond the text itself, like social connections) can boost detection accuracy. However, the why behind these performance gains and how these features influence the model's internal workings have remained largely unexplored. This paper aims to bridge this gap by providing a deeper understanding of the impact of user context on hate speech detection models.
Methods
The authors compare two main models:
- Text Model: A baseline model that only uses the tweet's content (represented as Bag of Words) for classification.
- Social Model: An enhanced model that incorporates the tweet's text, the author's vocabulary (derived from all their tweets), and the author's follower network (represented as a graph adjacency matrix).
To understand the impact of these features, they employ two key explainability techniques:
- Shapley Values: A method to quantify the contribution of each feature (e.g., individual words, user vocabulary, follower network) to a specific prediction. This helps identify which features are most influential.
- Feature Space Exploration (t-SNE): Visualizing the learned embeddings of tweets in a reduced-dimensional space. This allows researchers to see how the model clusters different types of content and how user features affect these clusters.
They also use artificially crafted tweets to test the model's behavior in specific scenarios, acting as a targeted behavioral analysis.
Datasets
The paper uses two publicly available datasets for hate speech detection:
- DAVIDSON (Davidson et al., 2017): Contains tweets labeled as "hate," "offensive," or "neither."
- WASEEM (Waseem and Hovy, 2016): Contains tweets labeled as "racism," "sexism," or "neither."
The authors note that the DAVIDSON dataset has a smaller proportion of "hate" tweets compared to WASEEM's "racism" and "sexism" classes.
Key Insights and Ideas
- User context is a significant driver of performance: The social model consistently outperforms the text-only model, especially on the WASEEM dataset, demonstrating that user features are not just noise but crucial for better detection.
- User features change how models learn: Beyond just improving accuracy, user features fundamentally alter the model's internal representation of the data. The feature space learned by the social model shows better separation of classes compared to the text-only model.
- Implicit hate speech benefits from context: The paper suggests that sexism, which can be more subtle and implicit, is harder to detect with text alone but shows significant improvement when user context is included.
- User features introduce new biases: While user features can mitigate biases present in the text itself (e.g., making the model less sensitive to certain words), they can also introduce new biases. For instance, a user's past behavior (being a "racist user") can heavily influence the classification of their tweets, potentially leading to unfair discrimination.
- Explainability is key to understanding bias: Techniques like Shapley values and t-SNE are vital for not only understanding performance gains but also for uncovering and diagnosing these new forms of bias.
Key Results
- Performance Gains: On the WASEEM dataset, the social model achieved a 4.3% overall F1 score improvement over the text model. On DAVIDSON, the improvement was smaller (1%), attributed to less available user data.
- Class-Specific Improvements: The social model showed a particularly strong improvement in detecting sexism on WASEEM.
- Feature Space Visualization: The text model's learned feature space for DAVIDSON showed racist tweets as sparse and mixed with offensive ones. The social model, however, showed better separation, with hateful tweets concentrated in smaller regions.
- Targeted Bias Detection: By crafting specific tweets and assigning them to different user profiles (racist, sexist, regular), the authors demonstrated how the social model's predictions change, highlighting its sensitivity to user history and the potential for bias. For example, an anti-black tweet was classified as racist when from a "racist user" but not when from a "regular user."
Limitations and Future Work
- Data Scarcity: The authors acknowledge that data scarcity, especially for context features, remains a significant challenge. Larger and more comprehensive datasets are needed to further improve understanding and bias mitigation.
- Model Simplicity: The models used were relatively simple to facilitate interpretability. Future work could explore these concepts with more complex, state-of-the-art architectures.
- Bias Mitigation: While the paper identifies new biases introduced by user features, further research is needed to develop effective strategies for mitigating these biases without sacrificing performance.
- Broader Application: The authors suggest that comparing recognition approaches beyond simple accuracy metrics is vital for the field to avoid potential harm to users. Their work provides a framework for such deeper analysis.
Summary updated