Abstract
This study presents a comprehensive multi-dimensional analysis of bias—encompassing gender, ethnicity, and age—within Vision-Language (VL) models. By introducing the Vision-Language Association Test (VLAT), the researchers quantify how biases from individual modalities are not only inherited but often amplified in multi-modal systems. Findings reveal that pre-trained VL models are significantly more likely to generate hurtful content than their text-only counterparts, with distinct stereotypical associations persisting across downstream tasks like Visual Question Answering (VQA).
Introduction
Vision-Language (VL) models, which integrate computer vision and natural language processing to perform tasks like image captioning and visual question answering (VQA), have seen rapid advancement. However, as these models become more integrated into real-world applications, the risk of encoded societal biases becomes a critical concern. While bias in Computer Vision (CV) and Natural Language Processing (NLP) is a well-documented "known-known," research into their intersection in VL models is still in its infancy. This paper addresses a critical gap by providing a multi-dimensional look at how these models perceive and categorize individuals based on visual and textual cues, moving beyond single-dimension studies that typically focus only on gender.
Methodology
The researchers split their investigation into two distinct stages to evaluate bias at different levels of model architecture:
- Pre-trained Model Analysis: The authors adapted the HONEST metric (originally designed for NLP) to a multi-modal context. They provided models such as ViLT and VisualBERT with an image of a person and a neutral text prompt (e.g., "The individual likes to..."). They then measured the frequency of "hurtful" sentence completions using the HurtLex lexicon, a dictionary of offensive and aggressive words.
- Downstream Task Analysis: To measure bias in practical applications, the researchers introduced the Vision-Language Association Test (VLAT). Inspired by the Word Embedding Association Test (WEAT), VLAT assesses the probability of a model associating specific demographic groups with target concepts (e.g., "math" vs. "arts") through VQA-style probing.
Datasets
The study utilized several key datasets:
- UTKFace: Sourced images representing diverse genders, ethnicities (White, Black, Asian, Indian, Other), and age ranges (1–14 to 65+).
- VQA 2.0, Visual Genome, and COCO: Served as the training foundations for the evaluated downstream models (BLIP, OFA, ViLT, and NLX-GPT).
- Custom Benchmarks: The authors created a 2,500-instance dataset for pre-training evaluation and a 24,000-instance dataset for VQA evaluation.
The Vision-Language Association Test (VLAT)
The VLAT serves as a standardized benchmark for quantifying stereotypical associations. By presenting a model with an image of a person and a neutral question, the researchers can observe if the model's response aligns with common societal stereotypes.
Key Insights and Results
The study uncovered several significant phenomena regarding how multi-modality influences model behavior:
The "Multi-modal Penalty"
One of the most striking findings is that the addition of visual data often increases a model's propensity for bias. For instance, VisualBERT exhibited nearly double the "hurtfulness" score of its text-only predecessor, BERT. This suggests that visual features can act as strong triggers for latent biases that remain dormant in text-only environments.
Visual Neglect and Stereotype Persistence
In some models, particularly VisualBERT, the researchers identified "visual neglect," where the model ignored the specific visual context of the image and replicated the same biased textual outputs regardless of the subject.
Furthermore, the models consistently demonstrated stereotypical associations. While some traditional tropes persisted (e.g., associating men with "Career" and women with "Family"), others shifted in the multi-modal context. Interestingly, models associated men with "Arts" and women with "Math," a departure from typical NLP-only findings, indicating that the visual modality introduces a unique stereotypical landscape.
Ageism in AI
The models displayed a clear bias regarding age, frequently associating younger individuals with "Pleasant" concepts and older individuals with "Unpleasant" ones. Quantitative results showed that pre-trained models generated hurtful content approximately 5% of the time when prompted neutrally, with higher rates for female and younger subjects.
VQA Bias Scores
In the VQA evaluation using the VLAT metric:
- BLIP emerged as the most biased model with a score of 51.5%.
- NLX-GPT was the most "fair" model, scoring 6.2%.
Limitations and Future Work
The authors acknowledge several constraints in the current study. The analysis was conducted exclusively on English-language models and was limited to a binary representation of gender due to the constraints of the UTKFace dataset. Additionally, demographic labels were estimated by an algorithm (DEX), which may introduce its own errors.
Future work aims to broaden the study to more languages and develop an automated bias detection pipeline that can be integrated into the standard release cycle of new VL models.
Conclusion
This research highlights the urgent need for multi-dimensional bias evaluation in Vision-Language models. By demonstrating that multi-modality can amplify harmful associations and introducing the VLAT framework, the authors provide a crucial foundation for developing more equitable and robust AI systems.
Summary updated