Attribute Aware Filter-Drop for Bias-Invariant Classification

Shruti Nagpal, Maneet Singh, Richa Singh, Mayank VatsaWorkshops @ CVPR 2020 · OpenReview

Abstract

The widespread applicability of deep learning based algorithms demands dedicated attention towards ensuring unbiased behavior. Biased feature learning (for or against a particular sub-group) might often result in unfair predictions. In order to address the above issue, this research proposes a novel Filter-Drop algorithm for learning unbiased representations. The proposed technique focuses on learning the features useful for predicting the biasing attribute (or the sensitive attribute), followed by their elimination while performing the primary classification task. To this effect, a multi-task network is trained, which prevents the features capturing the attribute variations from being used for the primary classification task. The efficacy of the proposed Filter-Drop technique is demonstrated on two facial analysis datasets: UTKFace dataset and FairFace dataset. The proposed technique achieves similar performance across different ethnicity groups while training with highly skewed training data as well.

Introduction: The Challenge of Algorithmic Bias

Deep learning models are prone to learning "spurious correlations." When training datasets are skewed—for instance, if a specific demographic group is overrepresented in certain contexts—models may inadvertently use sensitive attributes as shortcuts for prediction. This leads to algorithmic bias, where performance is high for majority groups but significantly degraded or unfair for minorities.

Figure 2: Distribution of images in ImageNet, highlighting significant geographic bias.

The goal of this research is to achieve bias-invariant classification, ensuring that a model's primary prediction remains mathematically independent of sensitive "bias co-variates" like ethnicity or age.

Methodology: Attribute Aware Filter-Drop

The core innovation is the Attribute Aware Filter-Drop algorithm. Unlike standard Dropout, which randomly removes neurons, Filter-Drop is a surgical regularization technique designed to "un-learn" specific biased features.

Figure 4: Detailed architecture of the Filter-Drop technique, showing the identification and removal of filters associated with the bias covariate.

The process involves four key steps:

  1. Multi-Task Learning: The network is trained simultaneously on a primary task (e.g., gender prediction) and a secondary task (predicting the sensitive attribute, such as ethnicity).
  2. Identifying Bias Filters: Following the final convolutional layer, Global Average Pooling (GAP) generates a feature vector. The weights of the secondary bias classifier are analyzed to identify which filters most strongly contribute to predicting the sensitive attribute.
  3. The "Drop": The top n filters identified as bias-heavy are zeroed out (masked) before the feature vector reaches the primary task's classifier.
  4. Feature Disentanglement: This forces the primary classifier to rely on the remaining filters, which are theoretically stripped of information regarding the sensitive attribute.

During inference, the bias-predicting branch is discarded, but the learned mask is retained to maintain bias-invariance.

Experimental Setup and Results

The method was evaluated using the UTKFace and FairFace datasets. To simulate real-world data scarcity, the authors created highly skewed training environments where minority group data was reduced to 10% of the majority.

Key Findings

  • Optimal Filter Removal: The authors identified a "Goldilocks" zone for filter dropping. Removing between 100 and 250 filters (out of 2048 in a ResNet-50) provided the best balance. Removing too few failed to eliminate bias, while removing too many (500+) stripped away essential discriminative features, hurting primary task accuracy.
  • Reduced Disparity: On UTKFace, Filter-Drop significantly narrowed the ~3% accuracy gap between ethnicities seen in traditional models.
  • Robustness to Skew: Even with highly skewed data, Filter-Drop maintained a consistent accuracy of approximately 94.6% across all groups, whereas traditional models showed inconsistent performance.

Conclusion and Future Directions

Filter-Drop offers a computationally efficient alternative to complex adversarial methods (like GANs) for feature disentanglement. By intentionally "un-learning" what a model should not know, it promotes fairer outcomes.

Future research may focus on automating the selection of the hyperparameter n and extending the technique to handle intersectional biases—where multiple sensitive attributes (e.g., age and race) must be addressed simultaneously.

Summary updated

Abstract

The widespread applicability of deep learning based algorithms demands dedicated attention towards ensuring unbiased behavior. Biased feature learning (for or against a particular sub-group) might often result in unfair predictions. In order to address the above issue, this research proposes a novel Filter-Drop algorithm for learning unbiased representations. The proposed technique focuses on learning the features useful for predicting the biasing attribute (or the sensitive attribute), followed by their elimination while performing the primary classification task. To this effect, a multi-task network is trained, which prevents the features capturing the attribute variations from being used for the primary classification task. The efficacy of the proposed Filter-Drop technique is demonstrated on two facial analysis datasets: UTKFace dataset and FairFace dataset. The proposed technique achieves similar performance across different ethnicity groups while training with highly skewed training data as well.
BibTeX
@inproceedings{nagpal2020attribute,
  author = {Shruti Nagpal and Maneet Singh and Richa Singh and Mayank Vatsa},
  title = {{Attribute Aware Filter-Drop for Bias-Invariant Classification}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=MGXHH6gMS7P},
}
Figure