Abstract
Generated texts from large pretrained language models have been shown to exhibit a variety of harmful, human-like biases about various demographics. These findings prompted large efforts aiming to understand and measure such effects, with the goal of providing benchmarks that can guide the development of techniques mitigating these stereotypical associations. However, as recent research has pointed out, the current benchmarks lack a robust experimental setup, consequently hindering the inference of meaningful conclusions from their evaluation metrics. In this paper, we extend these arguments and demonstrate that existing techniques and benchmarks aiming to measure stereotypes tend to be inaccurate and consist of a high degree of experimental noise that severely limits the knowledge we can gain from benchmarking language models based on them. Accordingly, we propose a new framework for robustly measuring and quantifying biases exhibited by generative language models. Finally, we use this framework to investigate GPT-3's occupational gender bias and propose prompting techniques for mitigating these biases without the need for fine-tuning.
Introduction
Measuring bias in Large Language Models (LLMs) is a critical yet challenging task. Existing benchmarks often fail to distinguish between correlation and causation. For instance, a model might associate "nurse" with "she" not just due to internal stereotypes, but because of the specific phrasing or "confounders" in the prompt template. Furthermore, many datasets rely on subjective human labeling for what constitutes a stereotype and suffer from linguistic noise in open-ended prediction spaces. To address these limitations, we introduce OCCUGENDER, a benchmark grounded in causal inference designed to isolate the true impact of occupational stereotypes on LLM outputs.
Methodology: A Causal Framework for Bias Measurement
The core of our approach applies Judea Pearl’s Causal Inference framework, specifically $do$-interventions, to LLM prompting. By treating the prompt template and the occupational stereotype as distinct variables, we can measure the direct effect of the latter on the model's gender predictions.
Eliminating Confounders
To ensure that the prompt's linguistic style does not bias the results, we utilize 50 diverse templates (e.g., "I have a friend who is a {job}...") generated independently of job titles. By iterating over every combination of template and job, we effectively "break" the causal link between the prompt's structure and the stereotype. This ensures that the measured effect comes directly from the stereotype, not the prompt's phrasing.
Dataset Construction
OCCUGENDER is built using objective employment data from the 2021 U.S. Bureau of Labor Statistics (BLS). We selected 20 male-dominated and 20 female-dominated occupations to serve as our test cases. Unlike previous benchmarks like StereoSet or CrowS-Pairs, OCCUGENDER satisfies key desiderata: it eliminates confounding, uses objective labels, operates in a small prediction space (next-token pronoun prediction), and includes non-binary gender options.
Analysis and Key Insights
Our evaluation of ten state-of-the-art models, including the Llama, Mistral, and Gemma families, revealed several critical insights into how LLMs handle occupational gender bias.
The Template Effect and Asymmetry of Bias
We observed a significant "template effect," where specific phrasing can override stereotypes. For example, active voice templates often triggered more male predictions regardless of the occupation, proving why template diversity is crucial for a fair test. Furthermore, bias is highly asymmetrical; models are much "stricter" with male stereotypes. In male-dominated jobs, models like Gemma-7B-Instruct predicted "male" with up to 100% probability. Conversely, for female-dominated jobs, the association was weaker, with models still predicting "male" a significant portion of the time.
Instruction Tuning: A Double-Edged Sword
Instruction tuning significantly impacts bias. While moving from base models to instruction-tuned versions (e.g., Llama-3 to Llama-3-Instruct) increased the likelihood of using diverse pronouns like "they," it also frequently amplified stereotypical associations for female-dominated jobs. This suggests a complex trade-off between promoting demographic inclusivity and reinforcing existing stereotypes.
Conclusion
While OCCUGENDER provides a more robust measurement of bias, challenges remain. LLMs exhibit extreme sensitivity to prompt phrasing, and current models still struggle with the reliable production of non-binary pronouns beyond "they." Furthermore, occupational gender ratios are culturally dependent, and our current study is limited to U.S. statistics. Future work will focus on expanding template diversity and exploring automated debiasing techniques that can effectively close the gap in stereotypical associations without the need for extensive fine-tuning.
Summary updated