Women Wearing Lipstick: Measuring the Bias Between an Object and Its Related Gender

Ahmed Sabir, Lluís PadróEMNLP 2023 · DOI

Introduction

Image captioning models frequently suffer from a "hallucination" problem: they often predict a person's gender based on surrounding contextual cues rather than direct visual evidence. For instance, a model might identify a person in a kitchen as a "woman" regardless of their actual appearance, simply because of the objects present in the scene. While previous efforts have focused on balancing datasets or training specific de-biasing classifiers, there has been a lack of standardized metrics to quantify the semantic "pull" that specific objects exert on gender predictions. This research, presented at EMNLP 2023, introduces the Gender Score, a diagnostic tool designed to measure and predict these biases in out-of-the-box models using a Belief Revision framework.

Methodology: Measuring Semantic Pull

The researchers propose a two-pronged approach to quantify the relationship between visual objects and gender labels.

1. Gender Object Distance

First, the authors measure the "semantic relatedness" between gender terms (e.g., man, woman, person) and various objects. This is achieved using word embeddings (such as GloVe) and sentence embeddings (such as SBERT). This metric provides a baseline for how closely a word like "lipstick" is associated with a gender label in a purely linguistic vacuum.

2. The Gender Score (Belief Revision)

The core innovation of this work is the Gender Score, which utilizes the psychological concept of Belief Revision. This framework acts as a "plug-in" metric for existing models. It begins with an initial hypothesis—a gender prediction generated by a Language Model (like GPT-2)—and then "revises" that belief based on the objects detected within the image.

The proposed framework for measuring gender bias, showing (a) word embedding distance, (b) sentence embedding distance, and (c-d) the Belief Revision process for calculating the Gender Score.

The Gender Score calculates the degree to which the presence of an object (the evidence) should shift the probability of a gender label (the hypothesis). If an object and a gender have high semantic similarity, the score reflects a stronger bias, allowing researchers to pinpoint which objects are driving model hallucinations.

Experimental Setup and Datasets

The study was validated using several gold-standard datasets:

  • COCO Captions: The primary dataset (120,000 images), known for its inherent gender imbalance (roughly 1:3 ratio of women to men).
  • Flickr30K: Used for broader validation of semantic distance results.
  • Karpathy Test Split: A standard subset of COCO used to evaluate seven state-of-the-art captioning models, including OSCAR, BLIP, and ViLBERT.

Key Insights and Results

The application of the Gender Score revealed several critical aspects of model behavior:

  • Bias is Object-Specific: Not all objects exert the same influence. While "bicycles" are relatively neutral, objects like "lipstick" or "dishwashers" show a massive skew toward one gender. Conversely, "motorcycles" and "skateboards" drive strong male-centric biases.
  • "Right for the Wrong Reasons": Models often correctly identify a person's gender simply because they have associated a specific object with that gender during training. The Gender Score exposes when a model relies on these contextual shortcuts rather than visual features.
  • Predictive Accuracy: The Gender Score proved more accurate at predicting gender bias than simple object-gender co-occurrence counting. It effectively balances the metric by accounting for the fact that many objects are naturally gender-neutral.
Examples of Gender Object Distance and Gender Score calculations for various visual contexts, illustrating how the presence of specific objects (tennis ball, umbrella, paddle) influences gender probability.

Limitations and Future Work

While the Gender Score provides a powerful diagnostic tool, it has limitations. Its accuracy is heavily dependent on the quality of the underlying word and sentence embeddings. Furthermore, the current study treats gender as a binary (man/woman), which is an oversimplification of human identity; the authors suggest future work should address gender as a spectrum. Additionally, the framework is sensitive to errors in the initial object detector (e.g., Faster R-CNN).

Despite these challenges, the Belief Revision framework offers a promising path forward. The researchers suggest it could be adapted to measure other forms of bias, such as racial or cultural stereotypes, providing a versatile tool for auditing AI systems without the need for expensive retraining.

Summary updated

BibTeX
@inproceedings{sabir2023women,
  author = {Ahmed Sabir and Lluís Padró},
  title = {{Women Wearing Lipstick: Measuring the Bias Between an Object and Its Related Gender}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.279},
}
Figure