Abstract
Multilayer Adversarial Bias Removal (MABR) is a novel framework designed to eliminate social biases in Natural Language Processing (NLP) models without requiring prior knowledge of specific bias types or access to protected demographic labels. By integrating bias detectors across multiple layers of a neural network, MABR identifies and neutralizes biased "shortcuts" through adversarial training. Experimental results on sentiment analysis and occupation classification tasks demonstrate that MABR often outperforms existing debiasing methods, including those with full access to sensitive attribute data, while maintaining high task performance.
Introduction
Machine learning models frequently exploit "spurious correlations"—shortcuts in training data that lead to biased predictions. For instance, sentiment analysis tools may incorrectly associate specific dialects with negative sentiment, or hiring algorithms may link professional roles with gendered language. Traditionally, mitigating these biases requires knowing exactly which biases to target and having access to protected attribute labels (e.g., race, gender). However, in real-world scenarios, biases are often unknown, and privacy regulations frequently restrict the collection of demographic data. MABR addresses this "blind" debiasing challenge by identifying bias through model behavior rather than explicit labels.
Methodology
The core philosophy of MABR is that bias manifests as a shortcut. If a simple auxiliary model can predict whether the main model will succeed or fail on a task without knowing the task itself, the main model is likely relying on biased features rather than semantic logic.
MABR introduces three primary components:
- Multilayer Bias Detectors: Detectors are placed at every layer of the encoder (e.g., BERT or RoBERTa). This design acknowledges that lower layers typically capture word-level biases, while higher layers capture abstract stereotypes.
- Adversarial Training with Gradient Reversal: Once detectors identify biased samples, a Gradient Reversal Layer (GRL) is employed. During backpropagation, the encoder is incentivized to modify internal representations such that bias detectors can no longer distinguish between biased and unbiased samples, effectively "unlearning" shortcut features.
- Hard Sample Identification: Samples where the main model fails are treated as "hard biased" samples, ensuring the debiasing process addresses both obvious stereotypes and subtle errors.
Experiments and Results
The framework was evaluated on two benchmarks: a sentiment analysis task involving African American English (AAE) and Mainstream US English (MUSE), and an occupation classification task using online biographies to predict professional roles while ignoring gendered contexts.
MABR was compared against "fairness-aware" methods (using labels) and "fairness-blind" methods. In sentiment analysis using BERT, MABR reduced the True Positive Rate (TPR) gap by 10.6% and improved statistical parity (Independence) by 3.6%, with only a 0.5% drop in accuracy. In occupation classification using RoBERTa, MABR outperformed state-of-the-art blind methods, reducing the TPR gap by 2.3% and Independence by 4.4%.
Discussion and Conclusion
A key finding is that bias is not monolithic; removing it at multiple levels makes the model more robust. Removing bias only at the final output layer often leaves "residue" in earlier layers that the model can still exploit. MABR demonstrates that demographic labels are not strictly necessary for achieving fairness; by focusing on model shortcuts, the framework discovers bias automatically.
While effective, MABR's performance is sensitive to the threshold ($\tau$) used to identify biased samples, requiring manual tuning. Additionally, the use of auxiliary classifiers at every layer increases computational overhead during training. Future research will explore applying this multilayer adversarial approach to generative Large Language Models (LLMs), where bias is more complex and harder to quantify.
Summary updated