Abstract
Deep learning models are often pruned to reduce size for edge deployment, but this process can inadvertently amplify hidden biases. We present a novel end-to-end framework for fairness-aware adversarial network pruning. By framing pruning as an adversarial game between a generator and a discriminator, our method identifies and removes biased and redundant neurons simultaneously. Experimental results on facial attribute datasets demonstrate that our approach significantly improves fairness metrics—by up to 50%—while maintaining high classification accuracy, even at high compression ratios.
Introduction
As deep learning models scale, pruning becomes necessary for deployment on resource-constrained devices like smartphones. However, recent studies indicate that standard pruning techniques can worsen biases inherited from training data. Current "debiasing" strategies typically occur before or after pruning, leading to a suboptimal trade-off between accuracy, model size, and fairness. We address this by proposing a joint optimization framework that treats fairness as a core objective during the pruning process.
Proposed Framework
Our method, Fairness-aware Adversarial Network Pruning, utilizes a two-player adversarial game to achieve balanced outcomes. The system consists of two primary components:
- The Generator ($G$): This is the neural network being pruned. It learns a binary mask to prune connections while maintaining high classification accuracy and minimizing bias.
- The Discriminator ($D$): This component attempts to predict sensitive attributes (e.g., gender) based solely on the Generator's output.
By training these components against each other, the Generator learns to eliminate connections that contribute to biased predictions. We specifically optimize for two major fairness metrics: Disparate Impact (DI), which ensures similar positive outcome rates across groups, and Equalized Odds (DEO), which ensures similar error rates.
Experimental Results
We evaluated our framework on the CelebA and LFW datasets, focusing on gender as a sensitive attribute. Our method consistently outperformed state-of-the-art pruning techniques like SGDP.
Key findings include:
- Significant Fairness Gains: Our method improved fairness by approximately 50% compared to standard pruning methods while maintaining comparable accuracy.
- The 80% Sweet Spot: On the CelebA dataset, we successfully pruned 80% of parameters while actually improving fairness metrics compared to the original, unpruned "dense" model.
- Superiority over Training from Scratch: The adversarial approach achieved a DEO of 0.2149, significantly better than the 0.5071 achieved by training a sparse network from scratch at the same compression level.
Key Insights
- The Fairness Lottery Ticket: We discovered that "fair sub-networks" exist even within randomly initialized, untrained networks. This extends the "Lottery Ticket Hypothesis" to the realm of ethics, suggesting that ethical sub-structures are inherent in large models.
- Joint Optimization: Unlike sequential methods, the adversarial approach allows the pruning mask to specifically target "biased neurons" and "redundant neurons" simultaneously.
- Flexibility: The framework is end-to-end and learnable, making it adaptable to various network architectures like ResNet, ShuffleNet, and MobileNet.
Conclusion
This work demonstrates that fairness and model compression are not mutually exclusive. By integrating fairness into the pruning process via adversarial learning, we can produce efficient AI models that are both accurate and ethically robust. Future work will explore scaling this approach to intersectional fairness and continuous sensitive attributes.
Summary updated