A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifiers

Valentin Barrière, Sebastian CifuentesEMNLP 2024 · DOI

Abstract

This study uncovers systematic nationality bias in popular "off-the-shelf" tweet classifiers used for sentiment, emotion, and hate speech detection. By swapping names in real-world tweets, researchers demonstrate that models significantly favor Western names. This bias is directly linked to the perplexity of the underlying language model—revealing that sequences a model hasn't seen frequently in training are judged more harshly.

Introduction

As NLP models move from research labs to production environments—moderating social media, screening CVs, or analyzing customer sentiment—the "fairness" of these models becomes a critical concern. While gender and racial biases are well-documented, nationality bias—specifically how a person's name can trigger different model behaviors—is less understood. Most bias detection relies on artificial templates (e.g., "I hate [NAME]"), which don't reflect the complexity of real-world language. This research seeks to quantify how much a simple name change influences a classifier's decision and whether this bias is a byproduct of the model's familiarity with certain names during its pre-training phase.

Methodology

The researchers employed a counterfactual perturbation approach to isolate name-based bias. The process began with Template Extraction, using 8,891 real English tweets and Named Entity Recognition (NER) to identify person names. These names were then replaced with common names from 194 different countries using a massive gazetteer of over 44,000 names.

The counterfactual perturbation pipeline, illustrating the flow from target-domain data to generated counterfactuals and probability discrepancy analysis.

The resulting variations were processed through four widely used "Twitter-RoBERTa" classifiers (Sentiment, Emotion, Hate Speech, and Offensiveness). To identify the "root cause" of the observed shifts, the authors calculated the Pseudo-Log-Likelihood (PLL) of the sentences. PLL measures how "likely" or "familiar" a sentence is to the model; high perplexity indicates the model finds the sequence surprising.

Key Results and Insights

The study establishes that perplexity isn't just a measure of linguistic fluency; it's a predictor of social bias. There is a direct link between how often a name appeared in the model's training data and how "positively" the model treats it. Names from English-speaking countries (UK, US, Canada) often acted as a buffer, whereas names from countries like Morocco or India triggered much higher rates of negative classifications.

Table showing the mean shifts in classification probabilities across sentiment, emotion, hate speech, and offensiveness for various countries.

Specific findings include:

  • Emotion Shifts: Replacing a name with a Moroccan name caused a staggering 60% increase in "Anger" predictions compared to the original text.
  • Hate Speech Detection: The presence of certain names changed hate speech detection results by up to 23%.
  • Sentiment Disparity: Names from Spain, Turkey, and India showed significant deviations in sentiment probability, consistently receiving fewer "positive" labels than their Western counterparts.
  • The "Familiarity" Penalty: In the sentiment model, there was a strong negative correlation between perplexity and positiveness. Essentially, the more "surprised" the model is by a name, the less it "likes" the person.

Discussion and Limitations

By comparing counterfactuals of the same sentence, the authors "subtracted" the influence of grammar, proving that the shift in model output was driven specifically by the identity associated with the name. However, the study notes several limitations. The method relies on the NER system's ability to find names; if the NER system itself is biased and fails to recognize non-Western names, those names are never tested. Additionally, the study focused on encoder-based models (RoBERTa). Future work should apply this unsupervised perplexity-based method to large generative models (LLMs) like GPT-4 or Llama to see if these patterns persist in more advanced architectures.

Summary updated

BibTeX
@inproceedings{barriere2024study,
  author = {Valentin Barrière and Sebastian Cifuentes},
  title = {{A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifiers}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.34},
}
Figure