TL;DR
Researchers introduce ViSAGe, a large-scale dataset covering 135 nationalities designed to audit visual stereotypes in text-to-image (T2I) models. Their analysis reveals a powerful "stereotypical pull," where models like Stable Diffusion default to offensive and narrow tropes—particularly for the Global South—even when explicitly prompted with neutral or counter-stereotypical terms.
What Problem Does This Paper Address?
While we know that T2I models like Stable Diffusion and DALL-E reflect societal biases, existing benchmarks for measuring these biases are often limited. They typically focus on Western contexts (like the US) or a handful of identity groups. Furthermore, most stereotype datasets are purely textual.
The authors argue that there is a fundamental gap between a textual stereotype (e.g., "Germans are practical") and a visual one (e.g., "Mexicans wear sombreros"). You can't easily "see" practicality in a generated image, but you can see a hat. To build safer models, we need a way to measure what these models actually depict across a global scale.
Methods
The researchers developed a multi-stage pipeline to bridge the gap between text and vision:
- Filtering for "Visual" Attributes: Starting with a massive list of textual stereotypes, they used human annotators to identify which attributes are actually "imageable." They narrowed down a list of nearly 2,000 attributes to 385 clearly visual ones.
- Image Generation & Annotation: They used Stable Diffusion v1.4 to generate images for 135 nationalities using neutral prompts (e.g., "a photo of a [nationality] person"). Human annotators then tagged these images for the presence of specific visual stereotypes and drew bounding boxes around them.
- Quantifying "Stereotypical Pull": To see if the models are "stuck" on stereotypes, they compared images generated from neutral prompts to those generated with explicit stereotypical (e.g., "a poor Bangladeshi person") and non-stereotypical (e.g., "a rich Bangladeshi person") prompts. They used CLIP embeddings and cosine similarity to measure how much the "default" output leans toward the stereotype.
Datasets
- SeeGULL: An existing textual stereotype resource containing ~7,000 stereotypes for 178 countries.
- ViSAGe (Visual Stereotypes Around the Globe): The new benchmark introduced in this paper. It includes 385 visual attributes and a dataset of 40,057 image-attribute pairs with human-verified annotations and bounding boxes.
Key Insights and Ideas
- The "Visual" Distinction: Not all stereotypes are created equal in the eyes of a T2I model. By filtering for visual concreteness, the authors provide a more objective way to audit image generators.
- The Default is the Stereotype: One of the most striking findings is that for many nationalities, the model's "neutral" representation is almost indistinguishable from its "stereotypical" one.
- Global South Disparity: The "stereotypical pull" isn't uniform. Models exhibit a much stronger tendency to revert to tropes when generating images of people from Africa, South America, and South East Asia compared to Western nations.
- Homogenization: The models often produce a "homogenized lens," where diverse individuals within a nationality are reduced to a single, often offensive, visual archetype.
Key Results
- 3x Likelihood: Stereotypical attributes are three times more likely to appear in generated images than random visual attributes.
- Extreme Bias: For some groups, the bias is staggering; images of Nigerians were 27 times more likely to contain stereotypical attributes than non-stereotypical ones.
- Widespread "Pull": In 121 out of 135 identity groups, the default representation had a higher similarity to the "stereotyped" prompt than the "non-stereotyped" one.
- Offensiveness: Depictions of identities from the Global South were rated as significantly more offensive, often focusing on themes of poverty or lack of "civilization."
Limitations and Future Work
The authors acknowledge several constraints:
- Subjectivity: Deciding what is "visual" can still be subjective, though they mitigated this with consensus-based human labeling.
- Model Scope: The study focused on Stable Diffusion v1.4; results might vary (though likely persist) in newer models like DALL-E 3 or Midjourney.
- Intersectionality: The current work focuses primarily on nationality. Future work needs to explore how these visual stereotypes intersect with gender, skin tone, and age.
- Source Material: The study is limited by the stereotypes present in the initial SeeGULL textual resource.
Content Warning: The paper and the ViSAGe dataset contain examples of offensive stereotypes generated by AI models.
Summary updated