TL;DR
Researchers have developed BALPORTRAITS, a controlled dataset of 9,000 AI-generated portraits, to audit implicit demographic bias in Vision-Language Models (VLMs). By using an "advertising-casting" pretext, they discovered that all 16 tested models—including frontier models like GPT-4o mini and Claude 3.5 Haiku—exhibit significant, context-dependent biases that often only surface at the intersection of age, gender, and ethnicity.
What Problem Does This Paper Address?
Measuring bias in modern VLMs is a game of cat and mouse. Direct questions about sensitive attributes (e.g., "Is this person likely to be a CEO?") often trigger safety filters and refusals. Furthermore, using "in-the-wild" images introduces confounding variables like clothing, background, and image quality, making it hard to isolate whether a model's judgment is based on a person's face or their surroundings.
The authors address this by shifting from direct classification to implicit association. They ask models to perform a "pretext" task: scoring how "fit" a portrait is for various advertising scenarios. This bypasses refusal mechanisms while revealing the underlying social hierarchies the models have internalized.
Methods
The core of the study is a "fitness-as-pretext" framework. Instead of asking for a person's profession, the researchers ask the VLM to score (from 1 to 10) how suitable a portrait is for an advertisement for a specific occupation, brand, or financial product.
To turn these raw scores into a meaningful metric of bias, they:
- Standardized Scenarios: Evaluated 100 occupations, 50 brands, 50 activities, and 50 banking/insurance products.
- Prestige Weighting: Assigned "prestige scores" to each item (e.g., "Chief Executive" has higher prestige than "Dishwasher"). These scores were validated by human panels.
- Rank-Based Analysis: Calculated a Degree of Bias (DoB), defined as the standard deviation of mean ranks across 24 demographic segments. This measures how much the model's "desirability" rankings deviate from a state of demographic equity.
Datasets
The authors introduced BALPORTRAITS, a dataset of 9,000 fictional portraits generated using gpt-image-1.
- Balance: It covers 24 segments: 4 ethnicities (African, Asian, Caucasian, Indian) × 2 genders × 3 age anchors (25, 40, and 55 years old).
- Control: All portraits use identical framing, neutral expressions, black t-shirts, and simple backgrounds to eliminate visual confounders.
- Privacy: Because the faces are fictional and passed identity-change verification against their real-world "seeds," the dataset avoids the privacy pitfalls of using real people's photos.
Key Insights and Ideas
- Bias is a Chameleon: A model might seem fair in one scenario but highly biased in another. For example, ethnicity is the dominant factor in brand rankings, while age is the primary driver of bias in banking and activity scenarios.
- The Intersectional "Aha": Single-attribute audits (just gender or just race) are insufficient. Many disparities only emerge at the intersection. For instance, in banking, the "female penalty" often widens significantly as the age of the portrait increases.
- Scaling Doesn't Solve Bias: Surprisingly, within model families, larger and more "capable" models often exhibited higher degrees of demographic dispersion than their smaller counterparts.
- The "Caucasian" Baseline: Across almost all models and scenarios, Caucasian portraits were consistently ranked in the top two ethnic segments and were never the most penalized group. Conversely, Indian portraits were systematically discriminated against across nearly all tasks.
Key Results
The audit of 16 VLMs yielded sobering results:
- Universal Bias: Every single model tested showed demographic dispersion significantly above the "equity line" (the level of variation expected by chance).
- Statistical Significance: On average, 73% of all pairwise segment comparisons (e.g., 25-year-old Asian Female vs. 55-year-old Caucasian Male) were statistically significant.
- Frontier Model Performance: Even highly aligned models like
GPT54M(GPT-4o mini) andCLAH45(Claude 3.5 Haiku) showed high bias, withCLAH45being among the most dispersed models in the study.
Limitations and Future Work
The authors acknowledge several limitations:
- Fictional Identities: While generated portraits provide control, they aren't "counterfactual edits" (changing one attribute of the exact same face), meaning some facial geometry cues might still be entangled with demographic labels.
- Broad Proxies: The four ethnic segments are broad appearance proxies and do not represent the full spectrum of human diversity or non-binary gender presentations.
- Prestige Subjectivity: Prestige is culturally dependent; the study primarily uses Western-centric brand and occupation lists.
Future work aims to extend this to more diverse cultural contexts and evaluate how these implicit biases translate into real-world harms in deployed recommendation and screening systems.
The data and code are available at https://github.com/afm215/BALPortraits.
Summary updated