Why AI Is WEIRD and Shouldn't Be This Way: Towards AI for Everyone, with Everyone, by Everyone

Rada Mihalcea, Oana Ignat, Longju Bai, Angana Borah, Luis Chiruzzo, Zhijing Jin, Claude Kwizera, Joan Nwatu, Soujanya Poria, Thamar SolorioAAAI 2025 · DOI

TL;DR

This paper argues that current AI development is heavily skewed towards a "WEIRD" (Western, Educated, Industrialized, Rich, and Democratic) demographic, leading to biased models and limited applicability. The authors propose a vision for truly inclusive AI, emphasizing the need for diverse data, models, evaluations, and developers across the entire AI pipeline.

What Problem Does This Paper Address?

The paper tackles the critical issue of representation and inclusivity in Artificial Intelligence. While AI, particularly Large Language Models (LLMs) and Multimodal Models (LMMs), has seen rapid advancements, much of the world's population, cultures, and knowledge systems are underrepresented in the data used to train these models, the models themselves, and the metrics used to evaluate them. This lack of representation leads to several problems:

  • Mistakes and Misconceptions: AI systems can perpetuate biases and misunderstandings, leading to incorrect outputs and even harm when applied to diverse contexts.
  • Limited Applicability: AI tools may not effectively serve or be understood by communities outside the dominant demographic.
  • Homogenization of Knowledge: AI models, despite their vast information capacity, can inadvertently erase or marginalize unique cultural knowledge and collective intelligence.
  • Developer Bias: A lack of diversity among AI developers means that their own cultural perspectives and blind spots can be embedded into the technology.

The authors draw a parallel to the "WEIRD" populations often overrepresented in psychological studies, arguing that AI development suffers from a similar "weird" coverage. They aim to shift the paradigm from AI that is exclusive to AI that is "for everyone, with everyone, and by everyone."

Methods

The paper doesn't introduce novel algorithms or techniques in the traditional sense. Instead, it presents a visionary framework and a call to action for making AI more inclusive. The authors analyze the entire AI pipeline and identify areas for improvement, proposing a set of desiderata and opportunities. Their "methods" are essentially a structured approach to identifying and addressing the limitations of current AI development, focusing on:

  • Analysis of the AI Pipeline: Deconstructing AI development into key stages: Data (Sources, Annotations), Models (Design, Performance), and Evaluations (Metrics, Benchmarks), as well as the People involved and the Incentives that shape the process.
  • Identification of Limitations: Pinpointing specific issues within each stage, such as data diversity gaps, model biases, narrow evaluation metrics, and skewed developer demographics.
  • Proposing Opportunities and Solutions: Suggesting concrete steps and research directions to overcome these limitations, often by drawing on existing research and advocating for new approaches. This includes advocating for participatory design, culturally sensitive annotation standards, and diverse evaluation benchmarks.

Datasets

The paper does not introduce new datasets. Instead, it discusses the limitations of existing datasets and the need for more diverse ones. It references various datasets and benchmarks used in AI research, often highlighting how they contribute to the problem of limited representation. For example, it mentions:

  • ImageNet: Used as an example of a benchmark that may not be a definitive standard and hasn't been designed with cultural representation in mind.
  • VALUE: Mentioned as a specialized benchmark that can handle languages spoken by diverse groups.
  • DADA: Another benchmark for dialect adaptation.

The authors emphasize the need for the development of culturally and linguistically diverse benchmarks that go beyond English-centric and Western-biased data.

Key Insights and Ideas

  • The "WEIRD" AI Problem: AI development, much like psychological research, is disproportionately influenced by Western, Educated, Industrialized, Rich, and Democratic (WEIRD) populations, leading to a narrow scope and inherent biases.
  • Cultural Brains vs. AI Homogenization: Human success stems from collective cultural learning, while current AI models can homogenize and erase diverse cultural knowledge.
  • Holistic Pipeline Approach: Inclusivity must be addressed across the entire AI lifecycle, from data collection and annotation to model design, evaluation, and the people involved.
  • Beyond Surface-Level Diversity: True inclusivity requires more than just adding a few diverse data points; it demands a fundamental shift in how AI is conceived, built, and evaluated.
  • The Importance of "By Everyone": Diverse developers are crucial for fostering innovation and ensuring AI systems are grounded in real-life contexts and address the needs of various communities.
  • Incentives Matter: Economic drivers, government policies, and philanthropic efforts significantly influence which AI applications are prioritized, often favoring dominant markets and languages.

Key Results

This paper is primarily a visionary and analytical piece, not an empirical study with quantitative results. Therefore, there are no specific numerical results to report. The "results" are the insights and recommendations derived from their analysis of the current state of AI development.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Scaling Diversification: Expanding cultural knowledge to encompass the vast diversity of global cultures and their many dimensions is an ongoing and significant challenge, especially for cultures with limited online representation.
  • Data Transparency and Documentation: While concepts like "datasheets for datasets" exist, more work is needed to define and interpret data licenses for AI usage and to adapt them to AI data and models.
  • Defining Inclusive Standards: Developing annotation standards that truly account for global diversity and subjectivity is a complex task.
  • Practical Application and Real-Life Grounding: Many AI research papers lack clear articulation of the problems they tackle and fail to connect with the lived experiences of diverse communities.
  • Balancing Profit and Fairness: Creating economic incentives that encourage companies to balance market-driven profits with the development of inclusive AI is a persistent challenge.
  • Government Support Inconsistency: Government support for inclusive AI is often inconsistent, requiring better decision-making systems to justify investments in fairness.
  • Philanthropic Engagement: Encouraging philanthropic support for AI development requires carefully crafted justifications highlighting long-term value.
  • Global Representation in Development: Fostering a diverse AI workforce and establishing equal research partnerships with communities are essential but challenging goals.

Future work should focus on implementing the proposed vision by developing concrete tools, methodologies, and frameworks that promote inclusivity across all aspects of AI development.

Summary updated

BibTeX
@inproceedings{mihalcea2025why,
  author = {Rada Mihalcea and Oana Ignat and Longju Bai and Angana Borah and Luis Chiruzzo and Zhijing Jin and Claude Kwizera and Joan Nwatu and Soujanya Poria and Thamar Solorio},
  title = {{Why AI Is WEIRD and Shouldn't Be This Way: Towards AI for Everyone, with Everyone, by Everyone}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i27.35092},
}
Figure