Mitigating Gender Bias Amplification in Distribution by Posterior Regularization

Shengyu Jia, Tao Meng, Jieyu Zhao, Kai-Wei Chang

Abstract

Machine learning models often amplify societal biases found in training datasets. While existing work primarily examines bias in final "hard" predictions, this research reveals that bias amplification is fundamentally present in the model's underlying probability distributions (posteriors). We propose a Posterior Regularization (PR) framework that constrains these distributions to match training set statistics. Our approach successfully eliminates bias amplification in the posterior distribution and reduces bias in final predictions by 30.9%, all while maintaining competitive model accuracy.

Introduction: The Depth of Bias

Data-driven models frequently inherit and exaggerate societal biases. A common phenomenon is "bias amplification," where a model predicts a majority class (e.g., "woman" for "cooking") at a higher rate than its frequency in the training data. Traditionally, this was attributed to the "winner-take-all" nature of classification. However, this paper demonstrates that bias is not merely a decision-making artifact but is deeply embedded in the model's confidence scores before a final label is even selected.

Visual Semantic Role Labeling (vSRL)

The study evaluates bias within the context of visual Semantic Role Labeling (vSRL). In this task, a model identifies an activity (verb) and the associated entities (roles) within an image, such as "agent," "item," and "place." Using the imSitu vSRL dataset, the authors tracked how gender ratios for various activities shifted from the training data to the model's internal probability distributions.

Methodology: Posterior Regularization

To address distributional bias, the authors employ Posterior Regularization (PR). Rather than retraining the model, PR acts as an inference-time constraint. It seeks a new probability distribution that remains close to the original model output while satisfying corpus-level constraints—specifically, ensuring that the predicted gender ratio for an activity does not deviate significantly from the training set ratio.

Figure 6: Bias in distribution before mitigation. Many activities show significant amplification, where the predicted bias exceeds the training set bias (points above the diagonal).

Key Results

The application of PR yielded significant improvements in fairness without compromising performance.

Figure 4: Bias in distribution after applying Posterior Regularization. The predictions are now calibrated to align closely with the training set distribution.

Distributional and Prediction Improvements

  • Eliminating Amplification: The method almost entirely removed bias amplification in the posterior distribution, reducing the average amplification from 3.2% to near zero.
  • Fairer Hard Decisions: By calibrating the underlying probabilities, the bias in final "top-1" predictions was reduced by 30.9%.
  • Accuracy Preservation: Model accuracy remained stable, shifting only marginally from 23.2% to 23.1%.
  • Constraint Satisfaction: The number of activities violating gender constraints dropped from 109 to just 5.

Conclusion

This research highlights that bias mitigation must look beyond final predictions and address the skewed confidence scores within a model's probability distributions. By regularizing these distributions, we can create models that are not only accurate but also more faithful to the underlying data distribution, providing a robust path toward fairer AI systems.

Summary updated

Abstract

Advanced machine learning techniques have boosted the performance of natural language processing. Nevertheless, recent studies, e.g., (CITATION) show that these techniques inadvertently capture the societal bias hidden in the corpus and further amplify it. However, their analysis is conducted only on models’ top predictions. In this paper, we investigate the gender bias amplification issue from the distribution perspective and demonstrate that the bias is amplified in the view of predicted probability distribution over labels. We further propose a bias mitigation approach based on posterior regularization. With little performance loss, our method can almost remove the bias amplification in the distribution. Our study sheds the light on understanding the bias amplification.
BibTeX
@inproceedings{jia2020mitigating,
  author = {Shengyu Jia and Tao Meng and Jieyu Zhao and Kai-Wei Chang},
  title = {{Mitigating Gender Bias Amplification in Distribution by Posterior Regularization}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.264},
}
Figure