TL;DR
This paper argues that current abusive language detection models often fail by ignoring the social context of online conversations, leading to the misclassification of benign speech from marginalized groups. They propose using subreddit embeddings to capture this community context, demonstrating that it significantly reduces false positives and improves the accuracy of abusive language classifiers.
What Problem Does This Paper Address?
Abusive language detection is a critical but challenging task. Existing models often rely heavily on lexical cues, which can lead to systemic bias. For example, they might flag African-American Vernacular English (AAVE) as offensive or misinterpret reclaimed slurs used by marginalized communities as genuinely derogatory. This can result in the censorship of productive conversations and the silencing of these groups. The core problem is that these models lack an understanding of the social environment in which language is used, failing to grasp nuances like sarcasm, humor, or the specific sociolinguistic norms of different online communities.
Methods
The authors introduce two main contributions:
- Community Clustering: They demonstrate that online communities (specifically, subreddits) can be clustered based on the nature of their support towards marginalized groups. This is visualized using UMAP on subreddit embeddings, showing distinct clusters for supportive (e.g., LGBTQ+, Black) and antagonistic (e.g., misogynistic, racist) communities.
- Subreddit Embeddings for Context: They leverage these community insights by creating subreddit embeddings. These embeddings are generated using a "node2vec-style" approach on a large-scale Reddit dataset, capturing co-occurrence patterns between users and subreddits. These embeddings are then used as an additional feature in abusive language detection models.
They evaluate their approach by comparing context-insensitive classifiers (Logistic Regression and BERT) with context-sensitive versions that incorporate subreddit information.
Datasets
- Slur-Corpus: The primary dataset used for training and evaluation is the "Slur-Corpus" by Kurrek et al. (2020). This corpus contains 40,000 human-annotated Reddit comments, each containing a slur, labeled as derogatory (DEG), appropriative (APR), non-derogatory non-appropriative (NDNA), or homonym (HOM). The corpus is nearly evenly split between derogatory and non-derogatory labels.
- Reddit Comments: A large dataset of publicly available Reddit comments (over 998K subreddits and 42.7M unique authors over 12 years) was used to construct the subreddit embeddings.
Key Insights and Ideas
- Community Polarity: Subreddits naturally cluster based on their stance towards marginalized groups, forming distinct supportive and antagonistic communities. This provides a quantifiable way to understand the social landscape of online discourse.
- Context is Crucial for Nuance: The paper strongly suggests that simply analyzing the text of a comment is insufficient. The social context, specifically the community it originates from, is vital for correctly interpreting potentially ambiguous language, such as reclaimed slurs or sarcasm.
- Misclassification Silences Voices: The misclassification of benign speech from marginalized groups as abusive is not just an accuracy issue; it actively censors productive conversations and harms the very communities these tools are meant to protect.
- Subreddit Embeddings as a Proxy for Context: Using embeddings derived from subreddit co-occurrence is an effective way to capture this community context, allowing models to learn the sociolinguistic norms of different online spaces.
Key Results
- Reduced False Positives: The addition of subreddit context significantly reduced false positives across all non-derogatory labels (NDNA, HOM, APR) for both Logistic Regression and BERT models.
- Improved Accuracy for Appropriative Language: The most notable improvement was for appropriative language, where the false positive rate decreased by almost 8% with the addition of context. An example cited is a comment originally misclassified as derogatory but correctly identified as appropriative when its subreddit context was considered.
- BERT Outperforms: BERT-based models generally outperformed Logistic Regression, as expected, but the addition of context provided benefits to both.
- Context Sensitivity: While most comments were context-insensitive (94%), the 1364 classification errors that were rectified by adding context represented a significant portion (56%) of the context-sensitive errors.
Limitations and Future Work
- Platform Specificity: The current work focuses on Reddit. The authors acknowledge that applying these findings to other platforms with different community structures might require adaptation.
- Neighborhood Context: While adding direct subreddit context improved performance, expanding to include "neighborhood context" (i.e., the five most similar subreddits) showed only marginal improvement and, in some cases, a slight increase in false positives for appropriative language. This is attributed to smaller communities having less representation in the dataset.
- Future Work: The authors suggest extending this work by incorporating community information into existing, platform-agnostic, ensemble detection frameworks. They also emphasize the need for future research to explore how community identity can be integrated into all stages of abusive language research.
Summary updated