FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

Diego A. B. Moreira, Alef Iury Siqueira Ferreira, Gabriel Oliveira dos Santos, Luiz Pereira, João Medrado Gondim, Gustavo Bonil, Helena de Almeida Maia, Nádia Félix F. da Silva, Simone Tiemi Hashiguti, Jefersson A. dos Santos, Hélio Pedrini, Sandra Avila

Introduction

Vision-language models (VLMs), such as OpenAI’s CLIP, have fundamentally changed the landscape of multimodal AI by aligning visual and textual representations in a shared embedding space. However, these models are typically trained on massive, uncurated datasets scraped from the internet, leading them to inherit and often amplify human prejudices. Biases regarding religion, nationality, and sexual orientation are frequently encoded into the model's weights, manifesting as harmful associations—for instance, linking specific demographic groups with concepts like "danger" or "terror."

Traditional methods for mitigating these biases often involve retraining models on curated datasets. However, retraining is prohibitively expensive, environmentally taxing due to high energy consumption, and often fails to address the nuances of non-English contexts. This paper introduces FairPIVARA, a post-processing algorithm designed to surgically remove discriminatory biases from CLIP-based models without requiring additional training or gradient updates.

Methodology: Surgical Bias Removal

FairPIVARA operates on the premise that bias is not uniformly distributed across a model’s embedding space but is instead concentrated in specific dimensions. By identifying and "muting" these dimensions, the researchers can de-bias the model while preserving its core functionality.

The FairPIVARA pipeline consists of four primary stages:

  1. Bias Scoring: The method calculates a bias score ($d$) by measuring the cosine distance between "class concepts" (e.g., images representing different religious groups) and "attribute concepts" (textual descriptions of "good" vs. "bad" traits).
  2. Harmful Dimension Identification: The algorithm iteratively evaluates which dimensions of the feature embeddings contribute most significantly to the observed bias.
  3. Surgical Removal via Mutual Information (MI): To ensure that removing a dimension does not degrade the model's "intelligence," the researchers employ a Mutual Information test. This ensures that the dimensions targeted for removal contain biased noise rather than critical information necessary for downstream classification tasks.
  4. Optimization: Through empirical testing, the authors determined that removing approximately 10% of the embedding dimensions (e.g., 54 out of 512 or 768) provides an optimal balance between fairness and task performance.

Experimental Setup

The researchers evaluated FairPIVARA using both standard English-centric models (OpenCLIP) and CAPIVARA, a CLIP-based model specifically adapted for the Portuguese language. This allowed for an investigation into how biases manifest and can be corrected in non-English and culturally specific contexts.

The study utilized several key datasets:

  • MMBias: A comprehensive dataset featuring 3,500 images across categories such as religion, nationality, disability, and sexual orientation.
  • Brazilian Nationality Set: A custom set of 250 images designed to evaluate bias within a Brazilian cultural framework.
  • Refined Concept Sets: The authors introduced 60 new "good" and "bad" descriptors in both English and Portuguese to establish a more neutral baseline for bias assessment.

To ensure the model remained effective for practical applications, performance was benchmarked using ImageNet-1K, CIFAR-100, and the ELEVATER suite, which covers 20 diverse image classification tasks.

Results and Discussion

The application of FairPIVARA yielded significant improvements in model fairness with negligible costs to accuracy.

Bias Mitigation

FairPIVARA achieved a 98% reduction in relative bias for OpenCLIP and a 97.9% reduction for the Portuguese CAPIVARA model. Qualitative analysis of image retrieval tasks further illustrated this shift. For example, when queried with the term "Fanaticism," the original model predominantly returned images associated with Middle Eastern demographics. Post-processing, the FairPIVARA-enhanced model returned a diverse array of images, including sports fans and global political protesters, reflecting a more neutral and accurate representation of the concept.

Performance Retention

A critical concern in de-biasing is the "tax" on accuracy. FairPIVARA demonstrated that bias can be removed with minimal impact on downstream performance. On ImageNet-1K, top-1 accuracy decreased by only 0.5 to 1.2 percentage points. This suggests that the dimensions responsible for societal bias are largely distinct from those responsible for object recognition and general semantic understanding.

Cultural Context and Fine-tuning

The study highlighted that fine-tuning models for specific languages (like Portuguese) can inadvertently reinforce existing biases if the fine-tuning data mirrors the cultural stereotypes of the original training set. FairPIVARA’s ability to work as a post-processing step makes it a versatile tool for various localized versions of VLMs.

Limitations and Conclusion

While FairPIVARA is highly effective, it is not without limitations. The process of identifying harmful dimensions is computationally intensive, taking approximately 14 hours in the researchers' setup (though this can be mitigated through parallelization). Furthermore, the definitions of "good" and "bad" are inherently subjective and culturally dependent, meaning the de-biasing process itself must be handled with care to avoid introducing the researchers' own biases.

In conclusion, FairPIVARA represents a significant step forward in responsible AI. By treating bias as a localized "noise" within the embedding space, it provides a scalable, environmentally friendly, and highly effective method for cleaning up multimodal models, ensuring they are fairer and more equitable for global users.

Summary updated

BibTeX
@inproceedings{moreira2024fairpivara,
  author = {Diego A. B. Moreira and Alef Iury Siqueira Ferreira and Gabriel Oliveira dos Santos and Luiz Pereira and João Medrado Gondim and Gustavo Bonil and Helena de Almeida Maia and Nádia Félix F. da Silva and Simone Tiemi Hashiguti and Jefersson A. dos Santos and Hélio Pedrini and Sandra Avila},
  title = {{FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models}},
  booktitle = {Workshops @ BMVC},
  year = {2024},
}
Figure