TL;DR
This paper reveals significant racial bias in popular hate speech and abusive language detection datasets. Even when trained on these datasets, machine learning models disproportionately flag tweets written in African-American English (AAE) as abusive, suggesting that these systems could unfairly penalize minority users.
What Problem Does This Paper Address?
The proliferation of social media has led to a surge in the development of automated systems designed to detect and flag hate speech and abusive language. The goal is to protect users, especially those belonging to vulnerable groups, from online harassment. However, a growing body of research highlights that machine learning models, particularly those trained on data, can inherit and even amplify societal biases. This paper specifically investigates racial bias in datasets used for hate speech and abusive language detection. The authors are concerned that if these biased systems are deployed, they could end up discriminating against the very communities they are intended to protect, leading to unfair penalties and silencing of minority voices.
Methods
The researchers employed a rigorous experimental design to uncover racial bias. They trained classifiers on five different hate speech and abusive language detection datasets. Crucially, they then evaluated these trained classifiers on two distinct corpora of tweets: one aligned with African-American English (AAE) and another with Standard American English (SAE). To quantify the bias, they used bootstrap sampling to estimate the proportion of tweets from each corpus that the classifiers assigned to negative classes (like hate speech or harassment). They then statistically tested whether these proportions differed significantly between the AAE and SAE corpora. They also conducted a follow-up experiment by conditioning their analysis on specific keywords (like "ngga" and "btch") to see if the bias persisted even when controlling for potentially indicative terms.
Datasets
The study utilized five existing datasets for training and evaluating their classifiers:
- Waseem and Hovy (2016): Collected tweets containing hateful terms, annotated by domain experts and reviewed by an activist feminist.
- Waseem (2016): Relabeled a subset of the above dataset and added new samples, using feminist and anti-racism activists as annotators.
- Davidson et al. (2017): Collected tweets using keywords from the Hatebase lexicon and had them coded by US-based crowdworkers, focusing on context.
- Golbeck et al. (2017): Selected tweets using keywords related to various forms of racism and sexism, with a binary harassment/non-harassment label.
- Founta et al. (2018): Constructed a large dataset designed to better approximate real-world abuse rarity, using crowdworkers and a four-class coding scheme.
For evaluation, they used a large dataset of 59.2 million tweets from Blodgett et al. (2016), which was geolocated and matched with demographic data. This allowed them to create "black-aligned" (likely AAE) and "white-aligned" (likely SAE) tweet corpora for their experiments.
Key Insights and Ideas
- Systematic Bias is Pervasive: The study found consistent racial bias across all tested datasets, indicating that the problem is not isolated to a single dataset but is a systemic issue in the field.
- AAE is Disproportionately Flagged: The core finding is that models trained on these datasets are significantly more likely to classify AAE tweets as negative (hate speech, harassment, etc.) compared to SAE tweets.
- Bias Persists Even with Keywords: Even when controlling for specific keywords often associated with negative classes (like "n*gga"), the racial disparities largely persisted, suggesting the bias is deeper than just the presence of certain terms.
- Context Matters, But is Ignored: The paper highlights that a "one-size-fits-all," context-independent approach to detecting abuse is problematic, as speech norms vary across communities, and models may unfairly penalize certain groups.
- Annotator Bias is a Key Driver: The study suggests that biases in annotators (whether experts, activists, or crowdworkers) are a significant source of bias in the training data.
Key Results
- In Experiment 1 (comparing AAE vs. SAE tweets without keyword conditioning), black-aligned tweets were assigned negative labels more frequently than white-aligned tweets in almost all comparisons.
- For example, the classifier trained on the Davidson et al. (2017) dataset showed large disparities, with approximately 5% of black-aligned tweets classified as hate speech compared to 2% of white-aligned tweets. Similarly, 17% of black-aligned tweets were predicted as offensive versus 6.5% of white-aligned tweets.
- The Founta et al. (2018) classifier showed substantial racial disparities, classifying black-aligned tweets as hate speech at 11% and abusive at 18%, compared to 6% and 8% for white-aligned tweets, respectively.
- In Experiment 2 (conditioning on keywords), while some disparities decreased, they often persisted. For instance, when conditioning on "b*tch," the Founta et al. (2018) classifier classified black-aligned tweets as hate speech at 2.7 times the rate of white-aligned ones.
Limitations and Future Work
The authors acknowledge several limitations:
- No Ground Truth for Race: The Blodgett et al. (2016) dataset, while a good proxy for AAE/SAE, doesn't provide definitive ground truth labels for the racial identities of authors.
- Potential for Corpus-Level Bias: They cannot entirely rule out that differences in the distribution of negative classes in the corpora themselves, rather than inherent bias, contribute to the results.
- Classifier and Feature Set Dependency: Results might vary with different classifiers or feature sets; more sophisticated models could potentially alleviate or exacerbate bias.
- Lack of Interpretability: The study did not delve into why models made specific predictions, making it hard to pinpoint which linguistic features of AAE are being associated with negative labels.
- Single Dimension of Bias: The research focused solely on racial bias, and further work is needed to assess biases related to other protected categories like gender.
Future work should focus on:
- Improving Data Collection: Developing more robust methods for collecting and labeling data that avoid over-sampling or under-representing specific communities.
- Understanding Annotator Bias: Investigating how to integrate annotator expertise and training to mitigate bias.
- Addressing Imbalanced Datasets: Paying more attention to baseline rates of abusive language usage across different populations and the challenges of imbalanced classification.
- Contextual Understanding: Developing detection systems that are sensitive to social and cultural contexts, moving beyond a "one-size-fits-all" approach.
Summary updated