Abstract
Large language models (LLMs) have demonstrated remarkable capabilities in simulating human behaviour and social intelligence. However, they risk perpetuating societal biases, especially when demographic information is involved. We introduce a novel framework using cosine distance to measure semantic shifts in responses and an LLM-judged Preference Win Rate (WR) to assess how demographic prompts affect response quality across power-disparate social scenarios. Evaluating five LLMs over 100 diverse social scenarios and nine demographic axes, our findings suggest a "default persona" bias toward middle-aged, able-bodied, native-born, Caucasian, atheistic males with centrist views. Moreover, interactions involving specific demographics are associated with lower-quality responses. Lastly, the presence of power disparities increases variability in response semantics and quality across demographic groups, suggesting that implicit biases may be heightened under power-imbalanced conditions. These insights expose the demographic biases inherent in LLMs and offer potential paths toward future bias mitigation efforts in LLMs.
Introduction
As Large Language Models (LLMs) are increasingly integrated into socially sensitive roles—such as digital clones, AI therapists, and non-player characters (NPCs)—their ability to simulate unbiased human behavior becomes critical. While previous research has identified explicit biases (such as racial or gender slurs), implicit biases are significantly more difficult to detect. These are the subtle, unconscious shifts in tone, perspective, or quality that occur when a model is prompted to adopt a specific persona.
A major gap in current literature is the isolation of demographic variables from the social structures they inhabit. In the real world, social interactions are governed by power disparities (e.g., a boss vs. an employee, a doctor vs. a patient). This paper investigates how these power dynamics intersect with demographic personas to influence LLM behavior, asking if social hierarchies amplify inherent AI biases.
Methodology
The researchers developed a systematic evaluation framework consisting of three main components:
1. Scenario and Persona Generation
Using GPT-4o, the authors created 100 diverse social scenarios across 10 contextual dimensions, including hiring, healthcare, and criminal justice. Half of these scenarios featured a clear power disparity, while the other half represented individuals of equal standing. Personas were assigned to the "Subject" (SUB) and the "Responder" (RES) across nine demographic axes: Age, Disability, Gender, Nationality, Physical Appearance, Political Stance, Race, Religion, and Socioeconomic Status.
2. Evaluation Metrics
To capture the nuances of implicit bias, the study utilized two primary metrics:
- Demographic Sensitivity (Semantic Shift): Measured using cosine distance between a demographically-prompted response and a "neutral" baseline response. A higher distance indicates the model is changing its behavior more significantly for that specific demographic.
- Response Quality (Preference Win Rate): An "LLM-as-a-judge" (GPT-4o mini) evaluated responses based on the HHH criteria (Helpful, Honest, Harmless).
- Implicit Bias via Variability: The researchers calculated the standard deviation of these metrics across demographic groups to observe how inconsistently the model treats different identities.
3. Models and Data
The study evaluated five models: GPT-4o mini, LLaMA-3.1 8B, Qwen-2 7B, Gemma-2 9B, and Mistral-0.3 7B. The dataset comprised 100 unique scenarios with 112 unique demographic combinations per scenario.
Key Insights
The "Default Persona" Discovery
By identifying which demographics caused the least semantic shift from the baseline, the authors inferred the LLMs' internal "default." The models essentially operate from the perspective of a middle-aged, able-bodied, native-born, Caucasian, atheistic male with centrist views. When prompted to adopt a persona outside this profile, the model must "move" further away from its baseline, often introducing bias.
Power as a Bias Multiplier
One of the paper's most striking findings is that power imbalances make the model's biases more pronounced. Variability in response quality and semantics increased significantly under power-disparate conditions. For instance, GPT-4o mini saw a 55.5% increase in win-rate variability when power dynamics were introduced, suggesting that LLMs struggle to maintain fairness when one persona holds authority over another.
Sensitivity vs. Quality
A high semantic shift—the model changing its tone significantly for a group—did not always correlate with better quality. While the "Buddhist" persona saw high semantic shifts and improved quality, the "Migrant Worker" persona experienced high shifts but lower quality. Furthermore, qualitative analysis revealed that models often adopted patronizing stereotypes; for example, an "Old" persona interacting with a "Young" person frequently used condescending language like "honey" or "my dear."
Results Summary
- Default Demographics: The lowest semantic deviations occurred for Middle-Aged (4/5 models) and Caucasian (3/5 models) personas.
- Quality Degradation: Lower win rates were consistently observed in interactions involving "Old" responders addressing "Young" subjects, or "Mentally Disabled" individuals responding to "Abled" subjects.
- Human Validation: The automated framework showed moderate agreement with human judges (Cohen’s Kappa of 0.447) and high agreement with the AI's rationale (4.117/5 Likert score), validating the "LLM-as-a-judge" approach.
Conclusion
This research unmasks the demographic biases inherent in LLMs and demonstrates how social hierarchies can exacerbate these issues. While the study identifies these patterns, it also highlights the need for future work on intersectionality (how overlapping identities affect bias) and the development of models that embed ethical reasoning directly into the generation process. By understanding the "default persona" and the impact of power dynamics, developers can better target bias mitigation efforts in future AI systems.
Summary updated