When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts

Jun Seong Kim, Ye Kyaw Thu, Javad Ismayilzada, Junyeong Park, Eunsu Kim, Huzama Ahmad, Na Min An, James Thorne, Alice OharXiv 2025 · DOI · arXiv

Introduction

As Multimodal Large Language Models (MLLMs) become increasingly integrated into global applications, their ability to navigate diverse cultural contexts is paramount. However, a critical question remains: do these models truly understand cultural artifacts, or do they rely on superficial demographic cues? Researchers have introduced MIXCUBE, a novel benchmark designed to expose "cultural shortcut" biases—where models prioritize a person's ethnicity over the actual object when identifying cultural elements like food or clothing.

The MIXCUBE Benchmark

The MIXCUBE (Mixed Culture Benchmark) addresses a significant gap in AI evaluation. While existing benchmarks often test "pure" cultural contexts (e.g., an East Asian person with East Asian food), MIXCUBE focuses on cultural mixture contexts. It investigates whether models use semantic proxies (the artifact's features) or demographic proxies (the person's ethnicity) for identification.

The dataset consists of 2,475 images across five countries: Azerbaijan (low-resource), Myanmar (medium-resource), and South Korea, the UK, and the US (high-resource). These images cover three categories: Food, Festivals, and Clothing.

Methodology: Isolating Ethnicity

To rigorously test for bias, the researchers developed a counterfactual image synthesis pipeline:

  1. Seed Collection: Gathering original images of cultural artifacts.
  2. Face Masking: Using the Segment Anything Model (SAM) to isolate facial features.
  3. Ethnicity Swapping: Employing Stable Diffusion inpainting to replace the original person with individuals representing four ethnic phenotypes: African, Caucasian, East Asian, and South Asian.
  4. Evaluation: Testing state-of-the-art models like GPT-4o, GLM-4v-Plus, and InternVL2.5-26B on country and cultural marker identification tasks.
An example of the demographic shortcut: an MLLM correctly identifies a Cornish Pasty when held by a Caucasian boy but misidentifies it as a Chinese dumpling when held by an East Asian girl.

Key Insights and Results

The study reveals several troubling trends in current MLLM performance:

The Demographic Shortcut

Models frequently exhibit an over-reliance on human features. When a person's ethnicity "matches" the cultural artifact's origin, accuracy is high. However, swapping the ethnicity leads to an average accuracy drop of 7.64%. In extreme cases, particularly with low-resource cultures, GPT-4o showed performance swings of up to 58%.

Resource Inequality and Robustness

High-resource cultures (US, UK, South Korea) are significantly more robust. Models can often identify an English Breakfast regardless of the person's ethnicity because they have been exposed to vast amounts of data from these regions. In contrast, low-resource cultures like Azerbaijan are often misclassified based on the person's appearance—for instance, Azerbaijani Plov might be labeled as "Chinese dumplings" if the person in the image appears East Asian.

The Fragility of Advanced Models

Interestingly, more "intelligent" models like GPT-4o proved to be the most sensitive to these changes. While GPT-4o achieved the highest absolute accuracy, it also suffered the most dramatic performance drops when ethnicities were swapped, suggesting its cultural knowledge is heavily anchored to ethnic stereotypes.

Conclusion

The findings from MIXCUBE highlight a fundamental weakness in how MLLMs learn culture. By relying on demographic shortcuts, these models risk perpetuating visual stereotypes and failing in multicultural settings. The researchers argue for more diverse training datasets that decouple cultural artifacts from specific ethnicities, ensuring that AI can recognize a Cornish Pasty or Kimchi regardless of who is enjoying it.

Summary updated

Abstract

In a highly globalized world, it is important for multi-modal large language models (MLLMs) to recognize and respond correctly to mixed-cultural inputs. For example, a model should correctly identify kimchi (Korean food) in an image both when an Asian woman is eating it, as well as an African man is eating it. However, current MLLMs show an over-reliance on the visual features of the person, leading to misclassification of the entities. To examine the robustness of MLLMs to different ethnicity, we introduce MixCuBe, a cross-cultural bias benchmark, and study elements from five countries and four ethnicities. Our findings reveal that MLLMs achieve both higher accuracy and lower sensitivity to such perturbation for high-resource cultures, but not for low-resource cultures. GPT-4o, the best-performing model overall, shows up to 58% difference in accuracy between the original and perturbed cultural settings in low-resource cultures. Our dataset is publicly available at: https://huggingface.co/datasets/kyawyethu/MixCuBe.
BibTeX
@misc{kim2025when,
  author = {Jun Seong Kim and Ye Kyaw Thu and Javad Ismayilzada and Junyeong Park and Eunsu Kim and Huzama Ahmad and Na Min An and James Thorne and Alice Oh},
  title = {{When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts}},
  year = {2025},
  eprint = {2503.16826},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.16826},
}
Figure