Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Eve Fleisig, Genevieve Smith, Madeline Bossi, Ishita Rustagi, Xavier Yin, Dan Klein

Introduction

While Large Language Models (LLMs) like ChatGPT are often praised for their "human-like" fluency, this fluency is frequently calibrated to a very specific type of human. A recent study from researchers at UC Berkeley reveals that ChatGPT exhibits significant linguistic bias, defaulting to "standard" varieties of English. When interacting with minoritized dialects, the model often produces responses that are stereotyping, demeaning, or unnaturally formal, effectively marginalizing billions of speakers.

Examples of ChatGPT's failure to appropriately mirror minoritized dialects, leading to exaggerated features or overly formal, 'robotic' responses.

Methodology

The researchers investigated linguistic discrimination, or "linguicism," through a two-phase study focusing on ChatGPT (GPT-3.5 and GPT-4).

Study 1: Feature Retention

The team prompted the models with informal text from ten English varieties, including Standard American English (SAE), Standard British English (SBE), African American English (AAE), and several global varieties such as Nigerian, Indian, Kenyan, and Singaporean English. They annotated responses to determine if the model retained "paradigmatic" features of those dialects, such as specific lexical items (e.g., "shamba" in Kenyan English) or grammatical structures.

Study 2: Human Evaluation

Native speakers of all ten varieties were recruited to rate model responses across nine dimensions, including warmth, respect, stereotyping, and comprehension. The study compared "baseline" responses to "imitative" responses, where the model was explicitly prompted to match the user's dialect style.

Results: Linguistic Erasure and the "Standard" Default

The study found that ChatGPT acts as a linguistic "homogenizer." While it retains nearly 80% of SAE features, it retains as little as 2% of features from varieties like Jamaican English. This effectively "translates" the user's dialect back into a standard American default, regardless of the input style.

This bias is strongly linked to training data availability. The researchers found a clear correlation ($R^2 = 0.61$) between a dialect's estimated speaker population and the model's ability to retain its features. Furthermore, the model exhibited a strong orthographic bias, aggressively converting British or dialect-specific spellings (e.g., analyse or favour) to American spellings (analyze, favor), even when responding to Standard British English.

Results: Human Impact and the Caricature Effect

When the model attempts to imitate a minoritized dialect, it often fails to achieve accuracy, instead falling into caricature. Native speakers rated responses to minoritized dialects as significantly more problematic than those to standard varieties:

  • 25% more demeaning
  • 19% more stereotyping
  • 15% more condescending
Average human ratings across nine dimensions comparing Standard American/British English (SAE/SBE) with eight minoritized varieties.

Even GPT-4, despite being generally "warmer" than its predecessor, showed an 18% increase in stereotyping when attempting to imitate dialects. Users noted that these attempts often felt "off," "robotic," or "cringeworthy." This is compounded by a significant comprehension gap; the model was 9% less likely to correctly comprehend inputs in minoritized dialects compared to standard ones.

Discussion and Conclusion

The findings suggest that LLMs risk reinforcing global power dynamics by treating non-standard speakers as "second-class" digital citizens. The "standard-as-default" bias is not merely a technical limitation but a form of baked-in linguistic discrimination. Future work is needed to determine if these biases persist in other major world languages and to develop models that respect linguistic diversity without resorting to harmful stereotypes.

Summary updated

BibTeX
@inproceedings{fleisig2024linguistic,
  author = {Eve Fleisig and Genevieve Smith and Madeline Bossi and Ishita Rustagi and Xavier Yin and Dan Klein},
  title = {{Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.750},
}
Figure