Abstract
While prior research on text-to-image generation has predominantly focused on biases in human depictions, we investigate a more subtle yet pervasive phenomenon: demographic bias in generated objects (e.g., cars). We introduce SODA (Stereotyped Object Diagnostic Audit), a novel framework for systematically measuring such biases. Our approach compares visual attributes of objects generated with demographic cues (e.g., "for young people") to those from neutral prompts, across 2,700 images produced by three state-of-the-art models (GPT Image-1, Imagen 4, and Stable Diffusion) in five object categories. Through a comprehensive analysis, we uncover strong associations between specific demographic groups and visual attributes, such as recurring color patterns prompted by gender or ethnicity cues. These patterns reflect and reinforce not only well-known stereotypes but also more subtle and unintuitive biases. We also observe that some models generate less diverse outputs, which in turn amplifies the visual disparities compared to neutral prompts. Our proposed auditing framework offers a practical approach for testing, revealing how stereotypes still remain embedded in today's generative models. We see this as an essential step toward more systematic and responsible AI development.
Introduction: The Hidden Bias in Objects
Most fairness research in AI focuses on the obvious: how models represent human faces, skin tones, and gender roles. However, text-to-image models are increasingly used to generate content for marketing, product design, and entertainment where humans are not the primary focus. This research argues that demographic bias is "quietly embedded" into non-human objects.
If a model consistently generates pink pastel items when asked for a "shirt for women," it is not merely being helpful; it is reinforcing narrow gender norms. This implicit bias can limit consumer choice and automate societal stereotypes within digital commerce and design. Because these biases are subtle and distributed across thousands of images, they are notoriously difficult to measure without a systematic approach.
Methodology: The SODA Framework
To address this challenge, the researchers introduced SODA (Stereotyped Object Diagnostic Audit), a four-step systematic framework designed to surface and quantify these hidden associations.
- Controlled Prompt Generation: The team created "Base" prompts (e.g., "a car, one product only, no people") and "Demographic-Conditioned" prompts (e.g., "a car for women").
- Image Generation: They audited three state-of-the-art models: GPT Image-1 (DALL-E 3), Imagen 4, and Stable Diffusion (SDXL) across five object categories (cars, laptops, backpacks, cups, and teddy bears).
- Automated Attribute Discovery & Extraction: To avoid the subjectivity and labor of manual labeling, the researchers utilized GPT-4o as a vision-language model. GPT-4o identified distinguishable attributes—such as car body type, headlight shape, or cup functionality—and classified them across the 2,700-image dataset.
- Statistical Measurement: The audit employed three novel metrics:
- BDS (Base vs. Demographic Divergence): Measures how much a demographic cue shifts the image away from the "neutral" version.
- CDS (Cross-Demographic Disparity): Measures the visual distance between different demographic groups (e.g., "men" vs. "women").
- VAC (Visual Attribute Concentration): Measures diversity; a high score indicates the model defaults to a single stereotypical look.
Visual Findings: From Sippy Cups to Pink Cars
The audit revealed that adding demographic cues triggers extreme, non-diverse visual patterns that often reflect ageist or gendered assumptions.
The "Neutral" Default
A striking finding was that images from "neutral" base prompts most closely resembled prompts for "middle-aged" and "White" groups. This suggests that AI models possess a "hidden default" that aligns with dominant demographic groups, treating other demographics as deviations from the norm.
The Sippy Cup Effect
Biases extended beyond simple color choices to fundamental object functionality. For the "elderly" demographic, models frequently shifted the nature of the object—turning standard cups into sippy cups or mugs with double handles. Similarly, laptops for the elderly were often depicted with simplified "educational" interfaces, reflecting ageist stereotypes regarding technological literacy.
Model-Specific Behaviors
Different models exhibited unique "tells" when responding to demographic prompts:
- GPT often embedded literal text (e.g., writing "Middle-Aged" on a cup) or national flags to satisfy the prompt.
- Imagen tended to encode bias through structural changes, such as giving "Black-targeted" cars gull-wing doors or leaving "elderly-targeted" backpacks unzipped.
- Stable Diffusion appeared to have lower bias scores, but this was attributed to poor prompt adherence; the model often ignored the "no people" instruction, creating a noisy dataset that appeared statistically diverse but failed to follow the core constraints.
Quantitative Analysis: Measuring the Gap
The statistical results confirmed the visual observations. The "Elderly" and "Women" groups showed the highest divergence from base prompts (BDS scores of 0.328 and 0.323, respectively), indicating that models significantly alter their output when these groups are mentioned.
Furthermore, Imagen showed extreme Visual Attribute Concentration (VAC of 0.662). In 83 specific test cases, the model reached a perfect 1.0 score, meaning every single generated image shared the exact same attribute—such as 100% of cars for women being rendered in red or pink. Color emerged as the most consistently biased attribute across all categories, with charcoal gray dominating for men and red/pink for women.
Conclusion: Toward Systematic Auditing
The SODA framework demonstrates that stereotypes are not just a human problem but are deeply baked into the objects our AI models generate. While automated extraction via GPT-4o is highly accurate (95.4%), future work must address the limitations of simple prompts and expand the audit to more complex scene compositions and diverse product categories like furniture or clothing.
Revealing these embedded stereotypes is an essential step toward more systematic and responsible AI development. By quantifying these biases, researchers and developers can move toward a future where a "car for everyone" truly represents the diversity of the global population.
Summary updated