Abstract
This study uncovers systematic nationality bias in popular "off-the-shelf" tweet classifiers used for sentiment, emotion, and hate speech detection. By swapping names in real-world tweets, researchers demonstrate that models significantly favor Western names. This bias is directly linked to the perplexity of the underlying language model—revealing that sequences a model hasn't seen frequently in training are judged more harshly.
Introduction
As NLP models move from research labs to production environments—moderating social media, screening CVs, or analyzing customer sentiment—the "fairness" of these models becomes a critical concern. While gender and racial biases are well-documented, nationality bias—specifically how a person's name can trigger different model behaviors—is less understood. Most bias detection relies on artificial templates (e.g., "I hate [NAME]"), which don't reflect the complexity of real-world language. This research seeks to quantify how much a simple name change influences a classifier's decision and whether this bias is a byproduct of the model's familiarity with certain names during its pre-training phase.
Methodology
The researchers employed a counterfactual perturbation approach to isolate name-based bias. The process began with Template Extraction, using 8,891 real English tweets and Named Entity Recognition (NER) to identify person names. These names were then replaced with common names from 194 different countries using a massive gazetteer of over 44,000 names.
The resulting variations were processed through four widely used "Twitter-RoBERTa" classifiers (Sentiment, Emotion, Hate Speech, and Offensiveness). To identify the "root cause" of the observed shifts, the authors calculated the Pseudo-Log-Likelihood (PLL) of the sentences. PLL measures how "likely" or "familiar" a sentence is to the model; high perplexity indicates the model finds the sequence surprising.
Key Results and Insights
The study establishes that perplexity isn't just a measure of linguistic fluency; it's a predictor of social bias. There is a direct link between how often a name appeared in the model's training data and how "positively" the model treats it. Names from English-speaking countries (UK, US, Canada) often acted as a buffer, whereas names from countries like Morocco or India triggered much higher rates of negative classifications.
Specific findings include:
- Emotion Shifts: Replacing a name with a Moroccan name caused a staggering 60% increase in "Anger" predictions compared to the original text.
- Hate Speech Detection: The presence of certain names changed hate speech detection results by up to 23%.
- Sentiment Disparity: Names from Spain, Turkey, and India showed significant deviations in sentiment probability, consistently receiving fewer "positive" labels than their Western counterparts.
- The "Familiarity" Penalty: In the sentiment model, there was a strong negative correlation between perplexity and positiveness. Essentially, the more "surprised" the model is by a name, the less it "likes" the person.
Discussion and Limitations
By comparing counterfactuals of the same sentence, the authors "subtracted" the influence of grammar, proving that the shift in model output was driven specifically by the identity associated with the name. However, the study notes several limitations. The method relies on the NER system's ability to find names; if the NER system itself is biased and fails to recognize non-Western names, those names are never tested. Additionally, the study focused on encoder-based models (RoBERTa). Future work should apply this unsupervised perplexity-based method to large generative models (LLMs) like GPT-4 or Llama to see if these patterns persist in more advanced architectures.
Summary updated