Stable Bias: Analyzing Societal Representations in Diffusion Models

Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, Yacine Jernite

Abstract: As machine learning-enabled Text-to-Image (TTI) systems are becoming increasingly prevalent and seeing growing adoption as commercial services, characterizing the social biases they exhibit is a first step to lowering their risk of discriminatory outcomes. This evaluation, however, is made more difficult by the synthetic nature of these systems' outputs; since artificial depictions of fictive humans have no inherent gender or ethnicity nor do they belong to socially-constructed groups, we need to look beyond common categorizations of diversity or representation. To address this need, we propose a new method for exploring and quantifying social biases in TTI systems by directly comparing collections of generated images designed to showcase a system's variation across social attributes -- gender and ethnicity -- and target attributes for bias evaluation -- professions and gender-coded adjectives. Our approach allows us to (i) identify specific bias trends through visualization tools, (ii) provide targeted scores to directly compare models in terms of diversity and representation, and (iii) jointly model interdependent social variables to support a multidimensional analysis. We use this approach to analyze over 96,000 images generated by 3 popular TTI systems (DALL-E 2, Stable Diffusion v 1.4 and v 2) and find that all three significantly over-represent the portion of their latent space associated with whiteness and masculinity across target attributes; among the systems studied, DALL-E 2 shows the least diversity, followed by Stable Diffusion v2 then v1.4.

Introduction

As Text-to-Image (TTI) systems like Stable Diffusion and DALL·E 2 move from research curiosities to commercial staples, a critical question looms: what kind of world are they showing us? Auditing bias in these generative models is notoriously difficult because they create "fictive humans" who have no inherent gender or ethnicity. Furthermore, these systems are complex assemblages of text encoders and diffusion engines, making it hard to pinpoint the origin of bias. This paper introduces a rigorous framework for auditing social bias in generative models by measuring how they represent gender and ethnicity across different professions.

Methodology

The researchers developed a two-pronged approach to "triangulate" bias by generating two massive datasets:

  1. Identities Dataset: Used prompts with explicit markers (e.g., "Photo portrait of a Black woman at work") to map the model's latent space for specific identities.
  2. Professions Dataset: Used neutral prompts for 146 different jobs (e.g., "Photo portrait of a CEO at work") based on U.S. Bureau of Labor Statistics (BLS) data.
The Stable Bias evaluation pipeline, showing the flow from prompt selection to cluster-based and text-based analysis.

The analysis framework utilizes multimodal techniques. Text-based analysis uses Image-to-Text systems to analyze the frequency of gendered terms in captions. Image-based analysis uses latent embeddings to cluster "Identities" images into distinct regions, then maps "Professions" images into these same regions to see which identities the model "chooses" by default.

Key Insights and Results

The study analyzed over 96,000 images across three major systems: Stable Diffusion (v1.4 and v2.0) and DALL·E 2.

The "Default" Human

When prompts are neutral, models exhibit a strong "gravity" toward specific identity regions. In DALL·E 2, over 40% of images for various professions fell into a cluster characterized by "White men," even for jobs that are demographically diverse in reality. Conversely, identity clusters associated with "Black" and "African American" prompts were consistently under-represented; in DALL·E 2, these represented only 0.1% of the generated profession images.

Model Comparisons and the Safety Paradox

The researchers found that Stable Diffusion v1.4 was the most representative of the three, though still significantly biased. Interestingly, Stable Diffusion v2.0—which included more aggressive filtering and "safety" measures—was often found to be less diverse than its predecessor, suggesting that current data cleaning methods may inadvertently prune diversity.

Gender and Ethnicity Ranks across various diffusion models, showing cumulative percentages compared to labor statistics.

As shown in the ranking analysis, DALL·E 2 showed the largest discrepancy compared to real-world data, with captions mentioning women 27% less than actual BLS statistics. Across all models, professions like "clerk" and "data entry keyer" saw a 50%+ under-representation of women.

Conclusion and Future Work

The authors provide a rigorous framework for quantifying social bias in TTI systems. However, they acknowledge limitations, including a Western-centric focus on U.S. demographics and the binary constraints of current VQA tools. Future research should move beyond these categories to explore intersections of age, religion, and disability on a global scale. The team has released the Diffusion Bias Explorer to allow the community to continue auditing new models.

Summary updated

Abstract

As machine learning-enabled Text-to-Image (TTI) systems are becoming increasingly prevalent and seeing growing adoption as commercial services, characterizing the social biases they exhibit is a necessary first step to lowering their risk of discriminatory outcomes. This evaluation, however, is made more difficult by the synthetic nature of these systems' outputs; since artificial depictions of fictive humans have no inherent gender or ethnicity nor do they belong to socially-constructed groups, we need to look beyond common categorizations of diversity or representation. To address this need, we propose a new method for exploring and quantifying social biases in TTI systems by directly comparing collections of generated images designed to showcase a system's variation across social attributes -- gender and ethnicity -- and target attributes for bias evaluation -- professions and gender-coded adjectives. Our approach allows us to (i) identify specific bias trends through visualization tools, (ii) provide targeted scores to directly compare models in terms of diversity and representation, and (iii) jointly model interdependent social variables to support a multidimensional analysis. We use this approach to analyze over 96,000 images generated by 3 popular TTI systems (DALL-E 2, Stable Diffusion v 1.4 and v 2) and find that all three significantly over-represent the portion of their latent space associated with whiteness and masculinity across target attributes; among the systems studied, DALL-E 2 shows the least diversity, followed by Stable Diffusion v2 then v1.4.
BibTeX
@inproceedings{luccioni2023stable,
  author = {Alexandra Sasha Luccioni and Christopher Akiki and Margaret Mitchell and Yacine Jernite},
  title = {{Stable Bias: Analyzing Societal Representations in Diffusion Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=JXn96DCJfU},
}
Figure