Diverse Yet Biased: Towards Mitigating Biases in Generative AI (Student Abstract)

Akshit SinghAAAI 2024 · DOI

TL;DR

Generative AI models often exhibit an Anglo-centric bias due to training on internet-sourced data. This paper proposes creating moderately sized, socially inclined datasets to mitigate these biases and train AI to generate more inclusive content, demonstrating this with an Indian-context text-to-image model.

What Problem Does This Paper Address?

Generative Artificial Intelligence (AI) has shown remarkable capabilities in content creation, but a significant challenge is the inherent bias present in these models. This bias often stems from the vast, internet-scraped datasets used for training, which reflect existing societal inequalities, particularly an Anglo-centric perspective. This leads to AI-generated content that perpetuates stereotypes and lacks diversity, with tangible consequences for societal equity, trust in information, and the psychological well-being of users. The paper aims to address this by developing methods to create more diverse and inclusive datasets for training generative AI.

Methods

The authors propose a methodology for constructing moderately sized, "socially inclined" datasets. These datasets are designed to rectify existing imbalances and train models to generate socially inclusive material. Specifically, they leverage Wikipedia as a source, known for its rigorous editing and verification processes, to ensure the absence of harmful information and the inclusion of detailed captions. To create theme-specific datasets, they conduct keyword searches within Wikipedia's image-caption pairs (from the WIT dataset) and associated web pages. For their demonstration, they focused on an "India" theme, selecting pairs where either the caption or webpage contained "India" and were in English. They then trained a transformer-based text-to-image generative AI model on this dataset to assess its ability to produce culturally specific material.

Datasets

  • WIT (Wikipedia-based Image Text Dataset): Used as the source for image-caption pairs.
  • IID (Indian-context Image caption Dataset): A newly created dataset, described as the first of its kind, containing over 250,000 images with captions specifically based on the Indian context. It covers a wide range of subjects including landmarks, individuals, artistic works, and flora/fauna.
  • IHD (Indian Heritage Dataset): Another newly developed dataset with a specific emphasis on Indian architecture.

Key Insights and Ideas

  • The power of curated, theme-specific datasets: The paper highlights that even moderately sized datasets, when carefully curated with a social inclination, can effectively mitigate biases present in larger, more general datasets.
  • Wikipedia as a reliable source for bias mitigation: Leveraging Wikipedia's inherent verification processes offers a promising avenue for obtaining cleaner, less biased data for training AI.
  • Demonstrating cultural context comprehension: The study shows that training generative AI on context-specific datasets can lead to a more accurate and nuanced representation of cultural elements, as seen in the distinction between Western and Indian beach imagery.
  • The need for proactive bias mitigation: The authors emphasize that addressing bias in generative AI is not just a technical challenge but also an ethical imperative, requiring transparency, accountability, and fairness-aware strategies.

Key Results

The paper presents preliminary findings from training a text-to-image generative AI model on their newly created Indian-context datasets (IID and IHD). A qualitative comparison with OpenAI's DALL-E is shown in Figure 1. When prompted with "beach," DALL-E generated images typically associated with Western beaches (white sand), while the model trained on the Indian dataset produced images reflecting Indian beaches (brown sand, due to mineral content). Similarly, for the prompt "village," DALL-E showed European-style villages, whereas the trained model depicted Indian villages with hut-shaped houses and mountains. This demonstrates the model's ability to capture and generate culturally specific imagery.

Limitations and Future Work

The authors acknowledge that the current generative AI model exhibits some blurriness in its output, attributed to the significant computational resources and time required for training. Future work will focus on:

  • Investigating methods to enhance image quality.
  • Exploring the potential of using smaller datasets to counterbalance biases in larger ones.
  • Extending the methodology to generate datasets for any particular subject matter.
  • Encouraging public and private sectors to adopt improved criteria for AI development and assessment, prioritizing equality and accessibility.

Summary updated

BibTeX
@inproceedings{singh2024diverse,
  author = {Akshit Singh},
  title = {{Diverse Yet Biased: Towards Mitigating Biases in Generative AI (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30512},
}
Figure