TL;DR
This paper introduces a novel data curation technique called "Repair" that addresses contextual bias in datasets, a common issue that leads to unfair AI models. By strategically selecting a subset of data that balances the co-occurrence of protected attributes (like gender) with other classes, their method significantly reduces model bias without sacrificing overall performance, even in challenging cross-domain and active learning scenarios.
What Problem Does This Paper Address?
Deep learning models, while powerful, often inherit and even amplify biases present in their training data. One insidious form of bias is "contextual bias," where certain attributes are disproportionately associated with specific contexts or co-occurring objects. For instance, in the COCO dataset, objects like skis or surfboards are more frequently associated with men than women. This can lead to models that perform poorly or unfairly when encountering these objects in different contexts or when associated with different demographic groups. While many existing approaches focus on modifying model architectures or training procedures to combat bias, this paper argues that fixing the data itself is a more fundamental and generalizable solution. They aim to create datasets that are contextually fair, ensuring that models trained on them are inherently less biased, regardless of the specific task or architecture.
Methods
The core of this paper is a data repair algorithm that leverages the coefficient of variation (cv) to measure and minimize contextual bias. The cv is used as an inequality measure for the representation of co-occurring classes with respect to a protected attribute.
Here's a breakdown of their approach:
- Problem Formulation: They define a binary composition matrix representing the presence of classes in images. For a protected class (e.g., gender), they focus on a subset of images and their co-occurring classes.
- Objective Function: They formulate an optimization problem to select a subset of data that minimizes the cv of co-occurring class representations. This effectively aims to make the co-occurrence distribution as uniform as possible.
- Supervised Fair Selection (Algorithm 1): In a supervised setting, they iteratively select samples that, when added to the current subset, minimize the cv. This is a greedy approach to curate a fair dataset.
- Active Learning for Fair Training (ALOFT) (Algorithm 2): For scenarios with unlabeled data, they adapt their fair selection strategy within an active learning framework. Crucially, they use pseudo-labels from a base model to construct the contextual information needed for fair selection, allowing them to select diverse and contextually balanced samples for annotation. This approach aims to build fair datasets from scratch.
Datasets
The authors evaluated their methods on several well-known datasets:
- COCO [26]: Used extensively for object detection and multi-label image classification tasks. They specifically created subsets focusing on "Cup" as a protected class and its co-occurring objects, as well as for gender bias analysis.
- OpenImages [21]: Used for cross-dataset evaluation of object detection models.
- ObjectNet [6]: Used for cross-domain generalization experiments, featuring objects in unusual contexts.
- Dollar Street: Another dataset used for cross-domain evaluation, highlighting socio-economic biases.
- CelebA [27]: Used for gender classification in an active learning setting, considering face attributes as co-occurring classes.
They did not introduce new benchmark datasets but rather focused on curating fair subsets from existing ones.
Key Insights and Ideas
- Data is King (for Fairness): The paper strongly advocates for data-centric approaches to bias mitigation, arguing that fixing the data is more effective and generalizable than solely relying on model-level interventions.
- Contextual Bias is Pervasive: They highlight how co-occurrence statistics in datasets can lead to significant biases, impacting model fairness even when individual object detection is accurate.
- Coefficient of Variation as a Fairness Metric: Using cv as a measure of inequality in co-occurring class representation is an elegant and effective way to quantify and minimize contextual bias.
- Pseudo-Labels for Active Learning Fairness: The innovative use of pseudo-labels from a base model in ALOFT allows for fair data curation even when ground truth labels are unavailable, bridging the gap between active learning and fairness.
- Generalizability of Fair Data: Models trained on their contextually fair datasets demonstrate improved performance and fairness not only on the original domain but also in cross-domain settings.
Key Results
- Reduced Contextual Bias: Their "Ours" method consistently achieved significantly lower cv scores across various budgets and datasets compared to random, ranking, and per-class rank selection methods. For example, at 10% data budget for "Cup" and its co-occurring classes, their cv was 0.0014, compared to 0.486 for random selection.
- Improved Fairness Metrics: In supervised settings, their method led to lower EoD (Disparity in Equalized Odds) scores, indicating a more balanced true positive rate across co-occurring classes. For instance, at 20% data, their EoD was 0.083, while random selection was 0.099.
- Maintained or Improved Accuracy: Despite reducing bias, their approach generally maintained or improved mAP (mean Average Precision) and F1 scores. For example, at 30% data, their mAP was 37.5, outperforming random (35.15) and ranking (36.25).
- Effective in Active Learning: ALOFT outperformed other active learning baselines in achieving low EoD scores while also achieving higher mAP, demonstrating its effectiveness in building fair models from unlabeled data.
- Cross-Domain Generalization: Models trained on their curated data showed improved average precision for detecting "Cup" on unseen datasets like OpenImages, ObjectNet, and Dollar Street, highlighting the robustness of their approach.
- Gender Bias Mitigation: They showed significant reductions in bias amplification (∆) and improvements in mAP and F1 scores when addressing gender bias in COCO.
Limitations and Future Work
- Computational Cost of Optimization: While their greedy approach is effective, the paper notes that more sophisticated algorithms could be used to solve the optimization problem (Equation 3) optimally.
- Focus on Specific Bias Types: The current work primarily addresses contextual co-occurrence bias and gender bias. Future work could explore fairness across other protected attributes and bias types.
- Combining Fairness with Diversity: The authors suggest extending their work to explore fairness combined with diversity in active learning settings, aiming for models that are both fair and robust to a wide range of scenarios.
- Scalability: While the method is presented as simple and effective, further investigation into its scalability for extremely large datasets might be beneficial.
Summary updated