Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models

Mahammed Kamruzzaman, Md. Minul Islam Shovon, Gene Louis KimACL 2024 · DOI

Abstract

While research into Large Language Model (LLM) bias has traditionally focused on overt categories like race and gender, subtler social biases remain pervasive. This study introduces a new benchmark to quantify biases related to age, physical attractiveness, institutional prestige, and nationality. Our findings reveal that state-of-the-art models like GPT-4 consistently mirror human "halo effects," associating high-status groups with positive personality traits.

Introduction

As LLMs are increasingly deployed in high-stakes scenarios—such as resume screening and performance evaluation—the risk of automating social inequities grows. Beyond well-documented ethnic and gender biases, "subtler" biases often go unchecked. These include the "beauty-is-good" stereotype or the assumption that graduates from prestigious universities are inherently more competent. This paper explores how these associations are encoded within generative models.

Methodology: SAI and ASA Directions

To quantify these biases, the researchers developed a template-based dataset designed for sentence completion tasks. The evaluation occurs in two primary directions:

  1. Stimulus to Attribute Inference (SAI): The model is presented with a social group (stimulus) and must select an appropriate personality trait (attribute).
  2. Attribute to Stimulus Association (ASA): The model is given a trait and must identify the most likely social group associated with it.
Examples of Stimulus to Attribute Inference (SAI) and Attribute to Stimulus Association (ASA) tasks used to measure subtle bias.

The strength of these biases is measured using Kendall’s $\tau$, a statistical coefficient correlating the social status of the stimulus with the valence of the chosen attribute.

Dataset and Categories

The benchmark consists of 11,940 test instances covering four dimensions:

  • Age: Comparing "young" (25-35) vs. "old" (60-70).
  • Beauty: Using descriptors like "attractive" vs. "unattractive."
  • Institutional Status: Comparing top-tier US universities against community colleges.
  • Nationality: Categorizing countries by GDP per capita (Rich vs. Poor).

Key Insights: The Halo Effect

The study confirms that LLMs encode social hierarchies rather than just objective facts. A prominent finding is the "Halo Effect," where models credit individuals from high-status groups (e.g., wealthy nations or elite schools) with positive traits like "creativity" or "reliability," despite no logical connection.

Beauty bias emerged as the strongest across all models. GPT-4, for instance, showed a near-perfect correlation in beauty-related SAI tests. Furthermore, Institutional bias persisted even when controlling for education level, with models favoring prestigious university students over community college students for positive attributes.

Comparative Results

The researchers evaluated four major models: GPT-4, PaLM-2, Llama-2-13B, and Mistral-7B.

Comparison of bias metrics across different categories for GPT-4 and Mistral in both SAI and ASA directions.

As shown in the results, proprietary models like GPT-4 often exhibit stronger subtle biases than smaller open-source models. For example, GPT-4's association of beauty with positive traits is significantly more pronounced than Mistral's. Interestingly, Mistral was an outlier in ageism, failing to show significant bias in certain settings, which suggests that training data and alignment techniques play a crucial role in these outcomes.

Conclusion and Future Work

LLMs are not neutral observers; they are reflections of the biased data they consume. While current debiasing efforts focus on race and gender, this research highlights the need to address subtler dimensions of social status. Future work should expand these benchmarks to non-English languages and investigate whether existing debiasing techniques effectively mitigate these specific social hierarchies.

Summary updated

BibTeX
@inproceedings{kamruzzaman2024investigating,
  author = {Mahammed Kamruzzaman and Md. Minul Islam Shovon and Gene Louis Kim},
  title = {{Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.530},
}
Figure