TL;DR
This paper introduces MIXCUBE, a new benchmark dataset designed to evaluate how well multimodal large language models (MLLMs) handle cultural nuances in mixed-ethnicity contexts. They found that MLLMs perform significantly worse on underrepresented cultures when the depicted ethnicity changes, highlighting a critical bias that needs addressing for fairer AI.
What Problem Does This Paper Address?
As AI models become more integrated into our globalized world, it's crucial they can understand and respond accurately to diverse cultural contexts. Imagine an MLLM identifying kimchi correctly whether it's being eaten by an Asian person or an African person. However, current MLLMs often rely too heavily on the visual cues of the person in an image, leading to misclassifications when the depicted ethnicity differs from the cultural norm. This paper investigates this "cultural bias" in MLLMs, specifically focusing on how their ability to recognize cultural items (like food, festivals, or clothing) is affected when the person in the image is of a different ethnicity. They aim to understand if this bias is more pronounced for cultures that are less represented in typical training data.
Methods
The core of this research is the creation of a new benchmark dataset called MIXCUBE. The authors collected images of food, festivals, and clothing from five countries and three categories. For each original image, they synthesized four new versions by replacing the person with individuals of different ethnicities (African, Caucasian, East Asian, South Asian) using an inpainting model. This allows them to test MLLMs on the same cultural item but with varying ethnic representations. They then evaluated three prominent MLLMs (GPT-4o, GLM-4v, and InternVL2.5) on two tasks: Country Identification (identifying the origin of the cultural item) and Cultural Marker Identification (naming the item itself).
Datasets
- MIXCUBE Dataset: A newly introduced benchmark consisting of 2.5k images across five cultures (Azerbaijan, Myanmar, South Korea, UK, US) and three categories (Food, Festivals, Clothing). It includes original images and synthesized versions with altered ethnicities.
Key Insights and Ideas
- Ethnicity Matters: Changing the ethnicity of the person in an image significantly degrades MLLM performance, even when the cultural item itself remains the same.
- Low-Resource Cultures Suffer Most: The accuracy drop is much more pronounced for cultures that are underrepresented in typical AI training data (e.g., Azerbaijan, Myanmar) compared to high-resource cultures (e.g., UK, US).
- Visual Similarity Can Deceive: For some foods, MLLMs might recognize them based on general visual similarities to dishes from neighboring regions, leading to correct food identification but incorrect country identification.
- The Need for Diverse Data: The findings strongly suggest that current MLLMs lack robust cultural awareness for underrepresented cultures, underscoring the critical need for more diverse and representative training data.
Key Results
- Across all tested MLLMs, country identification accuracy dropped by an average of 7.64% when the depicted ethnicity was altered.
- GPT-4o, despite being the best-performing model overall, showed accuracy differences of up to 58% between original and perturbed cultural settings in low-resource cultures.
- High-resource cultures (UK, US) showed stable performance across different ethnicities, with accuracy drops generally less than 15%.
- Low-resource cultures (Azerbaijan, Myanmar) exhibited significant sensitivity to ethnicity changes, with accuracy drops often exceeding 40% for certain models and categories.
- Cultural Marker Identification (naming the item) also showed similar sensitivity trends, with models like GPT-40 and InternVL dropping up to 24% in accuracy for certain food items when synthesized with South Asian ethnicity.
Limitations and Future Work
The authors acknowledge that minor inpainting artifacts in synthesized images could contribute to performance drops, rather than solely inherent model biases. They also note that the "South Asian" ethnicity is a broad generalization, and the synthesized appearances are not meant to be representative of any specific South Asian ethnicity. Future work could expand the number of MLLMs evaluated, increase the range of cultures and ethnic depictions, and explore more categories of cultural markers. They also suggest using multiple inpainting tools to average outputs and mitigate technical uncertainties.
Summary updated