Stable Bias: Evaluating Societal Representations in Diffusion Models

Sasha Luccioni, Christopher Akiki, Margaret Mitchell, Yacine Jernite

Introduction

As generative AI moves from research labs to commercial creative tools, the "black box" nature of these models poses a significant risk: they may quietly bake societal prejudices into our visual culture. Researchers from Hugging Face and Leipzig University present a novel framework for auditing social biases in Text-to-Image (TTI) systems like Stable Diffusion and DALL-E 2. By comparing AI-generated images of 146 professions against US labor statistics, they found that these models consistently under-represent marginalized identities and often exacerbate real-world gender and ethnicity gaps.

The Challenge of Auditing Synthetic Humans

Auditing these biases is technically challenging. Unlike real people, synthetic humans in AI-generated images have no inherent gender or ethnicity. Traditional evaluation methods often rely on human annotators to "guess" the identity of a generated person, which introduces a new layer of subjective bias. Furthermore, TTI systems are complex "assemblages"—including safety filters, text encoders (like CLIP), and diffusion backbones—making it difficult to pinpoint exactly where a bias originates. This paper seeks a more objective, reproducible way to quantify how these systems represent (or misrepresent) society.

A Two-Pronged Evaluation Methodology

To avoid the pitfalls of manual labeling, the authors proposed a two-pronged approach to evaluate three major systems: Stable Diffusion (v1.4 and v2) and DALL-E 2.

Methodology Overview
  1. Textual Analysis (Image-to-Text): They used secondary AI models (a captioner and a Visual Question Answering system) to describe the generated images. By analyzing the frequency of gendered terms (e.g., "man" vs. "woman") in these descriptions, they could quantify the models' "internal" associations for specific professions.
  2. Image-Space Clustering: The researchers used a non-parametric clustering method. They first generated an "Identities" dataset using explicit prompts (e.g., "Photo portrait of a Black woman at work"). They then mapped the embeddings of images from a "Professions" dataset (e.g., "Photo portrait of a carpenter at work") into the same latent space. By seeing which "identity clusters" the profession images fell into, they could measure representation without forcing a human to assign a label to a pixel.

Key Findings: Amplification, Not Just Reflection

The study utilized US Bureau of Labor Statistics (BLS) data as the "ground truth" for real-world demographics across 146 occupations. The results revealed stark disparities:

  • Gender Disparity: On average, across all professions, Stable Diffusion v1.4 mentioned women in 38% of captions, while DALL-E 2 dropped to 20%. Both are significantly lower than the BLS average of 47%. DALL-E 2 showed the highest discrepancy compared to real-world data.
  • Ethnicity Representation: Clustering analysis revealed that over 40% of images generated for the "Professions" dataset fell into clusters associated with "White men." Regions of the latent space associated with "Black women" were significantly under-represented across almost all professional prompts compared to their actual presence in the workforce.
  • The "Markedness" Trap: Certain professions like "doctor," "firefighter," and "pilot" showed high "markedness." Models almost always generated these with specific gendered or ethnic traits, likely because the training data is saturated with stereotypical depictions.
Clustering Analysis for Laboratory Technicians

The models don't just reflect society; they often "squash" diversity. For example, even in professions that are gender-balanced in reality, models tended to gravitate toward a "default" representation (often White men). Furthermore, the authors emphasize that we should evaluate TTI Systems as a whole, as bias might stem from safety filters or prompt enhancement techniques rather than just the diffusion model itself.

Limitations and Future Work

The authors are transparent about several constraints. The study relies on US-centric Census and BLS categories, which do not capture the global diversity of gender and ethnicity. Additionally, the secondary models used to evaluate the images have their own proxy biases. Because systems like DALL-E 2 are closed-source, researchers cannot control for hidden prompt engineering or filtering. Finally, while the study uses discrete categories for measurement, gender and ethnicity are fluid, multidimensional social constructs that AI currently struggles to represent accurately.

To foster ongoing research, the team has released interactive exploration tools allowing the community to audit new models as they emerge.

Summary updated

Abstract

As machine learning-enabled Text-to-Image (TTI) systems are becoming increasingly prevalent and seeing growing adoption as commercial services, characterizing the social biases they exhibit is a necessary first step to lowering their risk of discriminatory outcomes. This evaluation, however, is made more difficult by the synthetic nature of these systems' outputs; since artificial depictions of fictive humans have no inherent gender or ethnicity nor do they belong to socially-constructed groups, we need to look beyond common categorizations of diversity or representation. To address this need, we propose a new method for exploring and quantifying social biases in TTI systems by directly comparing collections of generated images designed to showcase a system's variation across social attributes -- gender and ethnicity -- and target attributes for bias evaluation -- professions and gender-coded adjectives. Our approach allows us to (i) identify specific bias trends through visualization tools, (ii) provide targeted scores to directly compare models in terms of diversity and representation, and (iii) jointly model interdependent social variables to support a multidimensional analysis. We use this approach to analyze over 96,000 images generated by 3 popular TTI systems (DALL-E 2, Stable Diffusion v 1.4 and v 2) and find that all three significantly over-represent the portion of their latent space associated with whiteness and masculinity across target attributes; among the systems studied, DALL-E 2 shows the least diversity, followed by Stable Diffusion v2 then v1.4.
BibTeX
@inproceedings{luccioni2023stable,
  author = {Sasha Luccioni and Christopher Akiki and Margaret Mitchell and Yacine Jernite},
  title = {{Stable Bias: Evaluating Societal Representations in Diffusion Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=JXn96DCJfU},
}
Figure