Abstract
The exceptional performance of Large Language Models (LLMs) often comes with the unintended propagation of social biases embedded in their training data. While existing benchmarks evaluate overt bias through direct term associations between bias concept terms and demographic terms, LLMs have become increasingly adept at avoiding biased responses, creating an illusion of neutrality. However, biases persist in subtler, contextually hidden forms that traditional benchmarks fail to capture. We introduce the Description-based Bias Benchmark (DBB), a novel dataset designed to assess hidden bias where bias concepts are hidden within naturalistic, subtly framed contexts in real-world scenarios. We analyze six state-of-the-art LLMs, revealing that while models reduce bias in response to overt bias, they continue to reinforce biases in nuanced settings.
Introduction
Modern Large Language Models (LLMs) have become experts at navigating traditional bias benchmarks by avoiding explicit stereotypical terms. If asked, "Who is bad at math, the man or the woman?", a state-of-the-art (SOTA) model is trained to recognize that associating "woman" with "bad at math" is a violation of safety guidelines. Consequently, models like GPT-4o achieve near-perfect scores on benchmarks like BBQ (Bias Benchmark for QA).
However, this research argues that such neutrality is often superficial. Real-world bias is rarely expressed through blunt labels; it operates through semantic-level associations. A model might avoid the phrase "bad at math" but still consistently associate a female character with "furrowing her brow in frustration and repeatedly erasing her work" while associating a male character with "methodically organizing notes to find an efficient solution." This paper introduces the Description-based Bias Benchmark (DBB) to prove that safety training often mitigates the symptoms of bias without addressing the underlying stereotypical logic.
Methodology
The researchers developed a three-step pipeline to create the DBB, focusing on naturalistic descriptions rather than raw terms:
- Concept Extraction: GPT-4o was used to extract "bias summaries" from existing datasets (like BBQ and StereoSet) and generate pairs of opposite concepts (e.g., "good at math" vs. "bad at math").
- Naturalistic Question Design: An LLM acted as a "story writer," creating a neutral context and two answer options that describe the concepts through behaviors, emotions, or traits without naming the bias explicitly.
- Subtle Demographic Descriptors: Direct identity terms (like "the man") were replaced with subtle descriptors (e.g., "a grandmother living in a nursing home" or "James, who was on the board of several charities").
The benchmark measures the probability difference ($S$) of a model choosing a specific behavioral description when only the demographic identity is changed. The final DBB contains 103,649 test instances across five categories: Age, Gender, Race/Ethnicity, Socioeconomic Status (SES), and Religion.
Results and Analysis
The evaluation of six SOTA models reveals a significant gap between overt and hidden bias performance.
The Capability Paradox
A striking finding is the "Capability Paradox": more advanced models often exhibit higher bias in DBB. While GPT-4o is the "cleanest" model on traditional benchmarks (scoring nearly zero bias on BBQ-ambig), it shows the highest bias on DBB. This is because advanced models have a superior ability to "read between the lines" and understand the hidden meaning in descriptions, which ironically allows them to apply stereotypes more consistently when they aren't explicitly told not to. For example, GPT-4o's BBQ score of -0.0008 contrasts sharply with its DBB bias score of 69.53.
Patterns of Hidden Bias
Across all models, Race and Religion categories consistently showed the highest levels of hidden bias, while SES typically showed the lowest. Furthermore, more advanced models like GPT-4o had much lower "refusal rates" compared to smaller models, suggesting that DBB is a high-quality benchmark that models can actually understand and engage with, unlike some older datasets with confusing options.
Conclusion and Future Work
The DBB demonstrates that models have essentially learned to pass "keyword-based" safety tests while embracing the same bias concepts when presented as descriptive narratives. Future work is needed to address the computational efficiency of the evaluation method, which currently requires multiple queries per question. Additionally, expanding the benchmark to include intersectional identities and multilingual contexts will be crucial for a more comprehensive understanding of how LLMs reflect hidden societal biases.
Summary updated