Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias

Justus Mattern, Zhijing Jin, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schölkopf

Abstract

Generated texts from large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics. This paper introduces a causal formulation for bias measurement in generative language models. Based on this theoretical foundation, we outline a list of desiderata for designing robust bias benchmarks and propose OccuGender, a benchmark designed to investigate occupational gender bias. We test several state-of-the-art open-source LLMs, including Llama and Mistral, finding that these models exhibit substantial occupational gender bias. We also discuss prompting strategies for bias mitigation and the generalizability of our framework. Code and data are available at: https://github.com/chenyuen0103/gender-bias.

Introduction

Measuring bias in Large Language Models (LLMs) is notoriously difficult due to linguistic "noise." Existing benchmarks often suffer from confounding factors: the specific phrasing of a prompt (the template) might accidentally trigger a gendered prediction regardless of the stereotype being tested. For example, a model might associate the word "confident" with a specific gender based on its training data, skewing results. Furthermore, many benchmarks rely on subjective human labeling and force models into binary gender choices. This research addresses these limitations by providing a mathematically grounded, causal approach to bias measurement.

Methodology: A Causal Approach

The authors utilize causal inference (specifically Pearl’s do-calculus) to isolate the effect of a stereotype on a model's gender prediction. In traditional benchmarks like StereoSet, the intent of the prompt creator acts as a confounder, influencing both the template and the stereotype, which creates a spurious path to the predicted gender.

Causal graph of existing benchmarks showing confounding between template and stereotype.

To resolve this, the authors propose treating the stereotype (e.g., "firefighter") as a variable to intervene upon while keeping the prompt template independent. This breaks the confounding link, as illustrated in the OccuGender causal graph.

Causal graph of the OccuGender benchmark, where interventions eliminate the confounding path.

The OccuGender Benchmark

OccuGender is designed to satisfy five key desiderata for robust bias measurement:

  1. No template confounding: Templates are independent of stereotypes.
  2. Objective labeling: Uses 2021 U.S. Bureau of Labor Statistics data to identify 20 male-dominated and 20 female-dominated occupations.
  3. Small prediction space: Focuses on predicting a few pronouns rather than open-ended text to minimize noise.
  4. Explicit and Implicit coverage: Tests both direct and indirect bias triggers.
  5. Non-binary inclusion: Includes "Diverse" (neutral/non-binary) gender categories.

Key Insights and Results

The authors tested ten state-of-the-art models, including Llama-3, Mistral, and Gemma, leading to several critical findings:

  • Extreme Occupational Bias: Models show a strong preference for binary genders aligned with societal stereotypes. For male-dominated jobs, Llama-3-8B-Instruct predicted "Male" with 96.2% probability in explicit prompts.
  • Asymmetry of Bias: Models were consistently more certain about male-dominated roles than female-dominated ones.
  • Instruction-Tuning Effects: While instruction-tuning is intended to align models with human values, it often amplifies stereotypical associations. However, it also introduced the only measurable awareness of non-binary genders; base models almost exclusively predicted binary genders, whereas instruction-tuned models like Gemma-2-9B-Instruct showed up to a 16.7% probability for "Diverse" predictions.
  • Template Sensitivity: Models often associate active voice with male genders. By using a diverse set of 50 templates, the authors average out these linguistic preferences to isolate the underlying occupational bias.

Conclusion and Future Work

This work demonstrates that occupational gender bias remains deeply embedded in state-of-the-art LLMs. While the causal framework provides a more accurate measurement tool by reducing template confounding, challenges remain. The benchmark is currently limited to English and U.S.-centric labor data. Future research should expand these causal methods to diverse cultural contexts and a broader range of gender-neutral linguistic expressions to ensure global applicability of bias mitigation strategies.

Summary updated

Abstract

Generated texts from large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics. These findings motivate research efforts aiming to understand and measure such effects. This paper introduces a causal formulation for bias measurement in generative language models. Based on this theoretical foundation, we outline a list of desiderata for designing robust bias benchmarks. We then propose a benchmark called OccuGender, with a bias-measuring procedure to investigate occupational gender bias. We test several state-of-the-art open-source LLMs on OccuGender, including Llama, Mistral, and their instruction-tuned versions. The results show that these models exhibit substantial occupational gender bias. Lastly, we discuss prompting strategies for bias mitigation and an extension of our causal formulation to illustrate the generalizability of our framework. Our code and data https://github.com/chenyuen0103/gender-bias.
BibTeX
@inproceedings{mattern2025causally,
  author = {Justus Mattern and Zhijing Jin and Mrinmaya Sachan and Rada Mihalcea and Bernhard Schölkopf},
  title = {{Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.281},
}
Figure