TL;DR
This paper demonstrates that incorporating social network information via Geometric Deep Learning (specifically GraphSAGE) significantly improves the accuracy and fairness of automated "hateful user" detection. By leveraging the context of a user's social neighborhood, the researchers were able to eliminate the common algorithmic bias that misclassifies African-American Vernacular English (AAVE) as hateful, achieving a 0% false positive rate for African-American users in their best-performing model.
What Problem Does This Paper Address?
Automated hate speech detection is a minefield of bias. Most existing systems focus on analyzing individual pieces of content (e.g., a single tweet). However, these models often struggle with polysemy—words that have different meanings depending on the speaker's identity and context. A notorious example is the term "nigga," which is frequently used innocuously or self-referentially within the African-American community but is often flagged as "toxic" by content-blind classifiers. This leads to a high False Positive Rate (FPR) for African-American users, effectively suppressing their speech.
Furthermore, social media platforms are shifting from simply moderating content to identifying "hateful users." This paper addresses the dual challenge of making these user-level classification systems both more accurate and fairer, specifically targeting the reduction of racial bias.
Methods
The researchers compared traditional machine learning baselines against Geometric Deep Learning (GDL).
- Race Inference: To evaluate fairness, the authors used a distantly supervised probabilistic topic model to identify users likely to be African-American based on their use of AAVE. This was followed by manual verification to ensure a high-quality subset for testing.
- Classification Models: They tested seven models:
- Baselines: Logistic Regression (LR), Support Vector Machines (SVM), and Artificial Neural Networks (ANN) using only textual and user-level features.
- Feature-Engineered ANN: An ANN that included manually engineered network features (like centrality measures).
- Geometric Deep Learning: Three variants of GraphSAGE (mean aggregation, max pooling, and self-attention). Unlike "shallow" embedding methods like node2vec, GraphSAGE is inductive—it learns functions to aggregate information from a node's neighborhood, allowing it to generalize to new, unseen users without retraining the entire graph.
The primary fairness metric used was predictive equality, which requires that the False Positive Rate be balanced across different demographic groups.
Datasets
The study utilized a Twitter retweet network dataset originally prepared by Ribeiro et al. (2018):
- Total Network: 100,386 users (nodes) and over 2.2 million edges.
- Annotated Subset: 4,972 users manually labeled as either "hateful" or "normal."
- Fairness Subset: 136 users identified and verified as African-American (AA) to test for disparate impact.
Key Insights and Ideas
- Context is King: The "aha moment" here is that a user's social network provides the missing context that text alone cannot. Because hateful users tend to be homophilous (they cluster together in dense, resilient networks), the model can learn to distinguish between a user using "edgy" language in a normal community versus a user embedded in a hateful cluster.
- Beyond Guilt-by-Association: While there is a risk of "guilt-by-association," the authors found that GDL actually reduced bias. By dynamically learning network representations, the model realized that African-American users, despite using certain keywords, did not share the network characteristics of hateful clusters.
- No Fairness-Accuracy Trade-off: Contrary to the common belief that fairness constraints hurt performance, the best-performing model (GraphSAGE with mean aggregation) was both the most accurate and the fairest.
Key Results
The quantitative results highlight a clear victory for Geometric Deep Learning:
- Overall Accuracy: GraphSAGE (mean aggregation) achieved the highest AUC of 90.8% and the best F1-Score of 57.5%.
- Fairness Breakthrough: As shown in the fairness evaluation (Figure 3), the baseline ANN without network features had a 20.3% False Positive Rate for African-American users. GraphSAGE (mean aggregation) brought this False Positive Rate down to 0.0%.
The confusion matrix for the best model (Figure 1) shows that while the model is excellent at detecting hateful users, it still tends to be conservative, with more false positives (492) than false negatives (126) across the entire dataset. However, the crucial finding is that these false positives are no longer concentrated in the African-American subset.
Limitations and Future Work
- Precision vs. Recall: While the AUC is high, the precision remains relatively low (~46%). The models are good at finding hateful users but still flag many normal users as hateful.
- Data Scarcity: The number of annotated hateful users (544) and African-American users (136) is small. The authors suggest that more annotated data would likely improve precision.
- Ground Truth Reliability: Qualitative error analysis revealed that a significant portion of "errors" were actually due to human annotation mistakes in the original dataset.
- Future Directions: The authors suggest exploring heterogeneous graphs (treating users and tweets as different types of nodes) and incorporating more advanced language models like BERT, which were not feasible for this specific dataset due to missing raw text for some users.
Summary updated