TL;DR
This paper investigates whether pruning deep neural networks using the "lottery ticket hypothesis" exacerbates fairness issues across demographic groups in NLP tasks. They introduce a metric called Fairness Sensitivity to Pruning (FSP) and find that while pruning generally increases group disparities, this effect can be partially mitigated by using Distributionally Robust Optimization (DRO).
What Problem Does This Paper Address?
Deep neural networks are often computationally expensive, leading to research into model compression techniques like weight pruning. The "lottery ticket hypothesis" suggests that dense, randomly initialized networks contain sparse subnetworks ("winning tickets") that can be trained in isolation to achieve comparable accuracy to the original dense network, but with fewer parameters. However, recent work has raised concerns that this pruning process might compromise the fairness of these models, meaning they perform differently across various demographic groups. While this has been explored in domains like image recognition, its impact on Natural Language Processing (NLP) models, which are crucial for many applications, remained largely unexamined. This paper aims to fill that gap by empirically evaluating the fairness implications of lottery ticket pruning in NLP.
Methods
The authors employ the iterative magnitude pruning (IMP) method, commonly associated with the lottery ticket hypothesis, to extract winning tickets. This involves repeatedly training a network, pruning a fraction of its weights based on their magnitude, and then rewinding the remaining weights to their initial values before retraining. They explore both layer-wise and global pruning strategies.
To quantify the impact on fairness, they introduce a novel metric called Fairness Sensitivity to Pruning (FSP). FSP measures how the disparity in performance (specifically, the F1-score) between different demographic groups changes as the pruning ratio increases. This is calculated by fitting a linear model to the logarithm of the pruning ratio and the min-max group-level disparity and taking the gradient.
They also investigate the effectiveness of Distributionally Robust Optimization (DRO), a technique designed to improve model fairness by minimizing the worst-case expected loss across a set of possible data distributions. They compare the FSP of models trained with standard empirical risk minimization (ERM) against those trained with DRO.
Datasets
The study utilizes two text classification datasets annotated with demographic information:
- Trustpilot Corpus: This dataset contains user reviews from the Trustpilot website across five countries (Germany, Denmark, France, UK, US). Reviews are annotated with sentiment (binary: positive/negative) and demographic attributes of the author, such as gender, age, and location. The authors preprocessed this data to create demographic groups based on combinations of gender, age, and NUTS regions for location.
- CivilComments: This dataset consists of comments annotated for toxicity, with a subset also annotated for protected attributes like gender, race, and religion. The authors binarized toxicity scores and created demographic groups based on combinations of race and gender.
For both datasets, they sampled test sets of varying sizes (100, 200, or 500 instances per demographic) and used an 80-20 split for training and validation. They also addressed class imbalance by downsampling the majority class in the training data.
Key Insights and Ideas
- Pruning generally increases group disparity: The core finding is that weight pruning, particularly at higher rates, tends to widen the performance gap between different demographic groups in NLP models. This suggests that the "winning tickets" might not be as fair as the original dense models.
- Global pruning is often more detrimental to fairness: The study observed that global pruning strategies tend to lead to higher group disparities compared to layer-wise pruning.
- Instability plays a role: The authors suggest that the increased group disparity observed with pruning is partly due to the inherent instability of the pruning process itself.
- DRO offers a partial solution: Distributionally Robust Optimization can, in some cases, mitigate the negative fairness impacts of pruning. However, this effect is not always consistent across all model architectures, datasets, and pruning strategies.
- FFNNs are more sensitive than LSTMs: Feed-forward neural networks (FFNNs) generally exhibited higher Fairness Sensitivity to Pruning (FSP) values compared to Long Short-Term Memory (LSTM) networks, indicating they are more susceptible to fairness degradation from pruning.
Key Results
- In 14 out of 16 combinations of FFNNs and LSTMs across the Trustpilot Corpus and CivilComments datasets, weight pruning led to increased group-level performance disparities (positive FSP values).
- Global pruning generally resulted in higher group disparities than layer-wise pruning.
- For FFNNs, DRO training showed a considerable reduction in FSP in some instances, for example, from 0.497 to 0.089 for layer-wise pruning on CivilComments.
- For LSTMs, DRO improved fairness in 5 out of 8 cases, reducing average FSP with both layer-wise and global pruning.
- The paper highlights that while DRO can improve fairness, the effect is not consistently significant across all experimental setups.
Limitations and Future Work
- Limited scope of datasets and tasks: The study focused on two text classification datasets and did not explore other NLP tasks like sequence generation or question answering.
- Inconsistent DRO benefits: While DRO showed promise, its effectiveness in mitigating pruning-induced fairness issues was not universal, suggesting further research is needed to understand when and why it works best.
- Model degeneration: In a few instances, heavy pruning led to model degeneration where models produced single-class predictions, which were excluded from analysis. This highlights a potential practical challenge.
- Future work could explore:
- Investigating other pruning techniques beyond magnitude pruning.
- Examining fairness across a wider range of demographic attributes and intersectional groups.
- Developing more robust pruning methods that inherently preserve fairness.
- Exploring the interplay between pruning, fairness, and other model properties like robustness to adversarial attacks or distribution shifts.
Summary updated