Abstract
Machine learning models often amplify societal biases found in training datasets. While existing work primarily examines bias in final "hard" predictions, this research reveals that bias amplification is fundamentally present in the model's underlying probability distributions (posteriors). We propose a Posterior Regularization (PR) framework that constrains these distributions to match training set statistics. Our approach successfully eliminates bias amplification in the posterior distribution and reduces bias in final predictions by 30.9%, all while maintaining competitive model accuracy.
Introduction: The Depth of Bias
Data-driven models frequently inherit and exaggerate societal biases. A common phenomenon is "bias amplification," where a model predicts a majority class (e.g., "woman" for "cooking") at a higher rate than its frequency in the training data. Traditionally, this was attributed to the "winner-take-all" nature of classification. However, this paper demonstrates that bias is not merely a decision-making artifact but is deeply embedded in the model's confidence scores before a final label is even selected.
Visual Semantic Role Labeling (vSRL)
The study evaluates bias within the context of visual Semantic Role Labeling (vSRL). In this task, a model identifies an activity (verb) and the associated entities (roles) within an image, such as "agent," "item," and "place." Using the imSitu vSRL dataset, the authors tracked how gender ratios for various activities shifted from the training data to the model's internal probability distributions.
Methodology: Posterior Regularization
To address distributional bias, the authors employ Posterior Regularization (PR). Rather than retraining the model, PR acts as an inference-time constraint. It seeks a new probability distribution that remains close to the original model output while satisfying corpus-level constraints—specifically, ensuring that the predicted gender ratio for an activity does not deviate significantly from the training set ratio.
Key Results
The application of PR yielded significant improvements in fairness without compromising performance.
Distributional and Prediction Improvements
- Eliminating Amplification: The method almost entirely removed bias amplification in the posterior distribution, reducing the average amplification from 3.2% to near zero.
- Fairer Hard Decisions: By calibrating the underlying probabilities, the bias in final "top-1" predictions was reduced by 30.9%.
- Accuracy Preservation: Model accuracy remained stable, shifting only marginally from 23.2% to 23.1%.
- Constraint Satisfaction: The number of activities violating gender constraints dropped from 109 to just 5.
Conclusion
This research highlights that bias mitigation must look beyond final predictions and address the skewed confidence scores within a model's probability distributions. By regularizing these distributions, we can create models that are not only accurate but also more faithful to the underlying data distribution, providing a robust path toward fairer AI systems.
Summary updated