Mitigating Gender Bias Amplification in Distribution by Posterior Regularization

Shengyu Jia, Tao Meng, Jieyu Zhao, Kai-Wei Chang

Abstract

Advanced machine learning techniques have boosted the performance of natural language processing. Nevertheless, recent studies show that these techniques inadvertently capture the societal bias hidden in the corpus and further amplify it. However, their analysis is conducted only on models' top predictions. In this paper, we investigate the gender bias amplification issue from the distribution perspective and demonstrate that the bias is amplified in the view of predicted probability distribution over labels. We further propose a bias mitigation approach based on posterior regularization. With little performance loss, our method can almost remove the bias amplification in the distribution. Our study sheds light on understanding bias amplification.

Introduction

Machine learning models are known to amplify societal biases present in their training data. For instance, if a training dataset features 30% more images of "women cooking" than "men cooking," a trained model might predict "woman cooking" 70% more often when evaluated. While prior research has focused on calibrating "hard" predictions (the final label chosen via argmax), this paper investigates whether bias amplification is rooted deeper within the model's underlying posterior probability distribution. The authors demonstrate that the distribution itself is biased and propose a method to fix it at this foundational level.

Methods

To address distributional bias, the authors move beyond simple prediction calibration and employ Posterior Regularization:

  1. Quantifying Distributional Bias: Instead of counting hard predictions, they calculate the expected bias toward a specific gender for a given activity based on the model's predicted probability distribution.
  2. Corpus-Level Constraints: They define a set of constraints ensuring that the bias in the model's posterior distribution on a test set does not exceed the training set's bias by more than a tiny margin ($\gamma$).
  3. Optimization: Using Kullback-Leibler (KL) divergence, they find the closest feasible distribution to the model's original, biased distribution that obeys these fairness constraints.
  4. Inference: The dual problem is solved using gradient-based methods (the Adam optimizer) to find optimal constraint weights, followed by Maximum A Posteriori (MAP) inference on the newly regularized distribution.

Datasets

The experiments are conducted on the imSitu vSRL (visual Semantic Role Labeling) dataset. The task involves predicting activities (verbs) and roles/objects (nouns) depicted in images. The authors filtered the dataset to focus on human-oriented verbs and images with labels indicating binary gender.

Example of the visual Semantic Role Labeling (vSRL) task in the imSitu dataset.

Key Results

The study reveals that overconfidence during training drives distributional bias. Before mitigation, 109 activities violated fairness constraints. Calibrating the posterior distribution yielded significant improvements:

  • Distributional Bias Removed: The posterior regularization technique successfully reduced the average bias amplification in the distribution from $0.032$ to $-0.005$, effectively eliminating it.
  • Top Prediction Bias Mitigated: By fixing the underlying distribution, the bias amplification in the hard top predictions was reduced by 30.9% (from $0.097$ to $0.067$).
  • Accuracy Maintained: The debiasing process did not degrade model utility; accuracy remained virtually identical (23.2% vs 23.1%).
  • Constraint Compliance: After regularization, only 5 constraints were violated.
Bias in predicted distributions after applying posterior regularization.

Conclusion

This work proves that bias lives in the probabilities, not just the final predictions. The empirical proof shows that bias amplification isn't just a quirk of the argmax decision boundary. By addressing bias at the distributional level through posterior regularization, models can be made fairer with negligible impact on accuracy. Future work will explore non-binary attributes and the theoretical reasons why deep learning models amplify these biases during the overfitting process.

Summary updated

Abstract

Advanced machine learning techniques have boosted the performance of natural language processing. Nevertheless, recent studies, e.g., Zhao et al. (2017) show that these techniques inadvertently capture the societal bias hidden in the corpus and further amplify it. However, their analysis is conducted only on models' top predictions. In this paper, we investigate the gender bias amplification issue from the distribution perspective and demonstrate that the bias is amplified in the view of predicted probability distribution over labels. We further propose a bias mitigation approach based on posterior regularization. With little performance loss, our method can almost remove the bias amplification in the distribution. Our study sheds the light on understanding the bias amplification.
BibTeX
@inproceedings{jia2020mitigating,
  author = {Shengyu Jia and Tao Meng and Jieyu Zhao and Kai-Wei Chang},
  title = {{Mitigating Gender Bias Amplification in Distribution by Posterior Regularization}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.264},
}
Figure