ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation

Akshita Jha, Vinodkumar Prabhakaran, Remi Denton, Sarah Laszlo, Shachi Dave, Rida Qadri, Chandan K. Reddy, Sunipa Dev

TL;DR

Researchers introduce ViSAGe, a large-scale dataset covering 135 nationalities designed to audit visual stereotypes in text-to-image (T2I) models. Their analysis reveals a powerful "stereotypical pull," where models like Stable Diffusion default to offensive and narrow tropes—particularly for the Global South—even when explicitly prompted with neutral or counter-stereotypical terms.

What Problem Does This Paper Address?

While we know that T2I models like Stable Diffusion and DALL-E reflect societal biases, existing benchmarks for measuring these biases are often limited. They typically focus on Western contexts (like the US) or a handful of identity groups. Furthermore, most stereotype datasets are purely textual.

The authors argue that there is a fundamental gap between a textual stereotype (e.g., "Germans are practical") and a visual one (e.g., "Mexicans wear sombreros"). You can't easily "see" practicality in a generated image, but you can see a hat. To build safer models, we need a way to measure what these models actually depict across a global scale.

Methods

The researchers developed a multi-stage pipeline to bridge the gap between text and vision:

  1. Filtering for "Visual" Attributes: Starting with a massive list of textual stereotypes, they used human annotators to identify which attributes are actually "imageable." They narrowed down a list of nearly 2,000 attributes to 385 clearly visual ones.
  2. Image Generation & Annotation: They used Stable Diffusion v1.4 to generate images for 135 nationalities using neutral prompts (e.g., "a photo of a [nationality] person"). Human annotators then tagged these images for the presence of specific visual stereotypes and drew bounding boxes around them.
  3. Quantifying "Stereotypical Pull": To see if the models are "stuck" on stereotypes, they compared images generated from neutral prompts to those generated with explicit stereotypical (e.g., "a poor Bangladeshi person") and non-stereotypical (e.g., "a rich Bangladeshi person") prompts. They used CLIP embeddings and cosine similarity to measure how much the "default" output leans toward the stereotype.
Figure 1: We identify ‘visual’ stereotypes in the generated images of the identity group by grounding the evaluations in existing textual stereotype benchmarks. Yellow boxes denote annotated visual markers of known stereotypes associated with the identity group in the image. We use Stable Diffusion (Rombach et al., 2022) to generate images and evaluate them using the stereotypes present in the SeeGULL dataset (Jha et al., 2023).

Datasets

  • SeeGULL: An existing textual stereotype resource containing ~7,000 stereotypes for 178 countries.
  • ViSAGe (Visual Stereotypes Around the Globe): The new benchmark introduced in this paper. It includes 385 visual attributes and a dataset of 40,057 image-attribute pairs with human-verified annotations and bounding boxes.

Key Insights and Ideas

  • The "Visual" Distinction: Not all stereotypes are created equal in the eyes of a T2I model. By filtering for visual concreteness, the authors provide a more objective way to audit image generators.
  • The Default is the Stereotype: One of the most striking findings is that for many nationalities, the model's "neutral" representation is almost indistinguishable from its "stereotypical" one.
  • Global South Disparity: The "stereotypical pull" isn't uniform. Models exhibit a much stronger tendency to revert to tropes when generating images of people from Africa, South America, and South East Asia compared to Western nations.
  • Homogenization: The models often produce a "homogenized lens," where diverse individuals within a nationality are reduced to a single, often offensive, visual archetype.
Figure 3: The distinction between visual and non-visual stereotypes is key to their evaluation framework.

Key Results

  • 3x Likelihood: Stereotypical attributes are three times more likely to appear in generated images than random visual attributes.
  • Extreme Bias: For some groups, the bias is staggering; images of Nigerians were 27 times more likely to contain stereotypical attributes than non-stereotypical ones.
  • Widespread "Pull": In 121 out of 135 identity groups, the default representation had a higher similarity to the "stereotyped" prompt than the "non-stereotyped" one.
  • Offensiveness: Depictions of identities from the Global South were rated as significantly more offensive, often focusing on themes of poverty or lack of "civilization."
Figure 2: Global distribution of visual stereotypes across countries. Depth of the color indicates the number of visual stereotypes. A few examples of visual stereotypes of some countries are shown in the figure.

Limitations and Future Work

The authors acknowledge several constraints:

  • Subjectivity: Deciding what is "visual" can still be subjective, though they mitigated this with consensus-based human labeling.
  • Model Scope: The study focused on Stable Diffusion v1.4; results might vary (though likely persist) in newer models like DALL-E 3 or Midjourney.
  • Intersectionality: The current work focuses primarily on nationality. Future work needs to explore how these visual stereotypes intersect with gender, skin tone, and age.
  • Source Material: The study is limited by the stereotypes present in the initial SeeGULL textual resource.

Content Warning: The paper and the ViSAGe dataset contain examples of offensive stereotypes generated by AI models.

Summary updated

Abstract

Recent studies have shown that Text-to-Image (T2I) model generations can reflect social stereotypes present in the real world. However, existing approaches for evaluating stereotypes have a noticeable lack of coverage of global identity groups and their associated stereotypes. To address this gap, we introduce the ViSAGe (Visual Stereotypes Around the Globe) dataset to enable the evaluation of known nationality-based stereotypes in T2I models, across 135 nationalities. We enrich an existing textual stereotype resource by distinguishing between stereotypical associations that are more likely to have visual depictions, such as `sombrero', from those that are less visually concrete, such as 'attractive'. We demonstrate ViSAGe's utility through a multi-faceted evaluation of T2I generations. First, we show that stereotypical attributes in ViSAGe are thrice as likely to be present in generated images of corresponding identities as compared to other attributes, and that the offensiveness of these depictions is especially higher for identities from Africa, South America, and South East Asia. Second, we assess the stereotypical pull of visual depictions of identity groups, which reveals how the 'default' representations of all identity groups in ViSAGe have a pull towards stereotypical depictions, and that this pull is even more prominent for identity groups from the Global South. CONTENT WARNING: Some examples contain offensive stereotypes.
BibTeX
@inproceedings{jha2024visage,
  author = {Akshita Jha and Vinodkumar Prabhakaran and Remi Denton and Sarah Laszlo and Shachi Dave and Rida Qadri and Chandan K. Reddy and Sunipa Dev},
  title = {{ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.667},
}
Figure