Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models

Abhishek Kumar, Sarfaroz Yunusov, Ali EmamiACL 2024 · DOI

As Large Language Models (LLMs) transition from research novelties to foundational components of hiring tools, creative assistants, and judicial aids, the implications of algorithmic bias have intensified. While existing research has largely focused on identifying overt toxicity and explicit stereotypes, recent work from Brock University suggests that a more insidious form of bias remains under-measured: the quiet, unconscious defaults that define what a model considers "normal."

The Dual Nature of Subtle Bias

Traditional bias benchmarks often rely on multiple-choice questions or toxicity filters. However, these methods fail to capture the nuanced preferences inherent in open-ended creative tasks. This research identifies two specific gaps in current evaluation frameworks:

  1. Representative Bias: The tendency of a model to generate "default" content that mirrors the experiences of dominant identity groups (e.g., assuming a generic "person" is a "man" unless specified otherwise).
  2. Affinity Bias: The tendency of a model, when acting as an evaluator, to prefer narratives or viewpoints that align with specific identities over others.

Methodology: RBS and ABS

To quantify these behaviors, the researchers developed a dual-metric approach using the Creativity-Oriented Generation Suite (CoGS), a benchmark comprising 12 diverse tasks (e.g., short stories, haikus, choreography) across 30 distinct themes and 8 identity groups.

The methodological framework for calculating the Representative Bias Score (RBS) via cosine similarity and the Affinity Bias Score (ABS) via LLM evaluation.
  • Representative Bias Score (RBS): This metric compares a model’s unprompted "default" output against outputs where the model is explicitly instructed to adopt a specific identity. By converting these outputs into vector embeddings and calculating cosine similarity, the RBS identifies which identity the default generation most closely resembles.
  • Affinity Bias Score (ABS): In this setup, the LLM acts as a judge, ranking several versions of the same task generated from different identity perspectives. The ABS measures the standard deviation of these preferences; a higher score indicates a significant favoritism toward specific groups.

Results: Mapping the "Bias Fingerprint"

The study evaluated three major models—GPT-4, LLaMA-2, and Mixtral—revealing that bias is not monolithic but rather manifests as a unique "fingerprint" for each architecture.

A conceptual overview of the identity axes (Gender, Race, Sexual Orientation) and the resulting radar chart illustrating a model's unique bias fingerprint.

Across the board, models displayed a systematic leaning toward "white," "man," and "straight" identities in their default generations. GPT-4, for instance, showed a moderate representative bias (RBS of 0.04) with "man" as its normative standard. Conversely, LLaMA-2 appeared more balanced in gender (RBS of 0.01) but exhibited higher variance in racial representation, often favoring Black and Asian identities in evaluations—likely a byproduct of aggressive safety fine-tuning.

Interestingly, some biases were task-specific. Models showed a marked affinity for Asian identities in the "haiku" task, likely reflecting the cultural association between the poetic form and its Japanese origins. Meanwhile, Mixtral emerged as the most equitable judge, maintaining the lowest ABS scores across sexual orientation axes.

Conclusion and Future Directions

The introduction of RBS and ABS provides a necessary yardstick for measuring the subtle normative standards embedded within LLMs. While the current study was limited to three identity axes (race, gender, and orientation), it lays the groundwork for broader audits including age, disability, and socioeconomic status. As AI continues to shape social narratives, these metrics offer a vital tool for developers to move beyond toxicity filtering toward true representational equity.

Summary updated

BibTeX
@inproceedings{kumar2024subtle,
  author = {Abhishek Kumar and Sarfaroz Yunusov and Ali Emami},
  title = {{Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.23},
}
Figure