Bias Unveiled: Investigating Social Bias in LLM-Generated Code

Lin Ling, Fazle Rabbi, Song Wang, Jinqiu YangAAAI 2025 · DOI

Abstract

As Large Language Models (LLMs) become integral to software development, the risk of "hard-coded" social bias grows. This paper introduces Solar, a framework designed to detect and mitigate demographic prejudices in LLM-generated code. Using a new benchmark of 343 human-centered tasks, the study finds that popular models frequently embed discriminatory logic based on race, gender, and age. However, iterative feedback through Solar can reduce these biases by up to 90%.

Introduction

While extensive research has audited LLMs for toxic text, the "silent" bias within generated code remains largely unexplored. As developers increasingly utilize tools like GitHub Copilot or ChatGPT to scaffold software for hiring, loan approvals, or healthcare, there is a significant risk that these automated systems will inherit and programmatically enforce social prejudices present in training data.

Existing benchmarks, such as HumanEval, focus primarily on functional correctness—whether the code runs—rather than fairness—whether the code treats individuals equally. Previous attempts to measure code bias often relied on explicitly "judgmental" prompts. This research addresses more insidious, neutral prompts that result in biased logic, such as a career-recommendation function that explicitly excludes certain gender identities.

The Solar Framework

To address this challenge, the researchers developed Solar, a black-box evaluation framework inspired by metamorphic testing. The fundamental principle is that changing a single demographic attribute (e.g., from "male" to "female") while keeping all other variables constant should not alter the code's output. Any divergence in logic indicates the presence of bias.

Overview of the Solar framework for detecting and mitigating code bias.

Solar operates through a three-stage pipeline:

  1. Prompt Generation: High-level task definitions are transformed into structured Python code prompts using a Domain-Specific Language (DSL).
  2. Bias Testing: The framework generates executable test cases that instantiate "Person" objects with varying sensitive attributes (Race, Age, Religion, etc.) and monitors if the LLM-generated logic branches differently.
  3. Mitigation: Solar utilizes test results as feedback in an iterative dialogue with the LLM, prompting the model to refine its logic until fairness criteria are met.

SocialBias-Bench Dataset

The study introduces SocialBias-Bench, a comprehensive dataset comprising 343 social problems across seven critical categories: social benefits, university admissions, employee development, health programs, professional licenses, hobbies, and occupations. These tasks represent real-world scenarios where software decisions directly impact human lives. The scenarios were generated with the assistance of GPT-4o and manually verified for correctness.

Analysis of Code Bias

The investigation revealed that bias is often "hard-coded" into the logic. For instance, models might generate explicit if statements excluding specific groups, such as if self.gender != 'transgender'.

Key Results

The researchers evaluated four prominent models: GPT-3.5-turbo, codechat-bison, CodeLlama-70b, and Claude-3-haiku. The results, summarized in the table below, indicate that all models exhibit significant bias.

Code Bias Scores (CBS) across different demographic dimensions for major LLMs.

GPT-3.5-turbo was found to be the most biased, with an overall Code Bias Score (CBS) of 60.58%. Across all models, age, gender, and employment status were the most frequently targeted demographics.

Insights on Model Behavior

  • Correctness vs. Fairness: A model can achieve high functional correctness while remaining deeply biased. The researchers propose Pass@attribute to measure if code relies on relevant variables (e.g., skills) rather than sensitive ones (e.g., race).
  • Temperature and Role-Play: Higher temperature settings generally increased bias in GPT-3.5, though this was not universal. Surprisingly, "Positive Role-Play" (instructing the model to be fair) sometimes increased bias, likely because the model over-focused on the sensitive attributes mentioned in the instruction.

Conclusion and Future Work

The iterative prompting strategy proved highly effective for mitigation; GPT-3.5's bias score dropped from 60.58% to 8.77% after three rounds of feedback. While SocialBias-Bench is a major step forward, future work will expand the dataset to more programming languages and explore bias in complex, multi-file software architectures. The researchers have open-sourced their framework to encourage further community-driven fairness testing.

Summary updated

BibTeX
@inproceedings{ling2025bias,
  author = {Lin Ling and Fazle Rabbi and Song Wang and Jinqiu Yang},
  title = {{Bias Unveiled: Investigating Social Bias in LLM-Generated Code}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i26.34961},
}
Figure