Introduction
Machine learning models often rely on spurious correlations, particularly in sensitive domains like toxicity detection. For instance, a model might incorrectly associate identity terms (e.g., "transgender") with toxicity because those terms frequently appear in toxic contexts within training data. While Counterfactual Data Augmentation (CDA) attempts to mitigate this by swapping identity terms, it often overlooks hidden correlations with secondary attributes, such as the intent of a comment (e.g., a directed attack vs. a descriptive statement). This paper introduces RDI (Reweight-Direct-Indirect), a methodology that explicitly models these secondary attributes to build more robust and fair NLP models.
The RDI Methodology
To address the limitations of standard augmentation, the RDI framework uses a context-aware fine-tuning approach designed to disentangle primary attributes (identity) from secondary attributes (intent). RDI consists of three core components:
- Reweighting (R): This component uses inverse propensity weighting to balance the prevalence of secondary attributes across different primary groups, addressing sampling bias in the original dataset.
- Direct Constraints (D): Following standard Counterfactual Token Fairness (CTF), this penalizes the model if its prediction changes when only the primary attribute (e.g., gender or race) is swapped.
- Indirect Constraints (I): The core innovation of this work. In tasks like toxicity detection, the model should be sensitive to the secondary attribute (intent). RDI adds a constraint that maximizes the model's sensitivity to changes in the secondary attribute, ensuring it learns to distinguish between a directed slur and a descriptive discussion.
Experimental Results
The RDI framework was evaluated on the Jigsaw Kaggle Toxicity dataset and coreference resolution tasks using OntoNotes and Winogender.
Toxicity Detection
RDI improved "sliced accuracy" (performance on specific demographic groups) by 2-7% compared to standard BERT and existing CDA methods. For the "black" identity slice, the AUC increased by 6.98%. Unlike many bias-mitigation tools that sacrifice overall performance for fairness, RDI manages to improve both robustness and core model accuracy by providing a richer, more balanced training signal.
Coreference Resolution
In structural tasks, RDI significantly reduced gender bias. The "Gendered Correlation" (a measure of how much a model relies on gender stereotypes for professions) dropped from 0.37 in vanilla BERT to 0.11 with RDI. This demonstrates the framework's versatility across both classification and structural NLP tasks.
Discussion and Limitations
While RDI provides a robust framework for disentangling attributes, it currently relies on manually defined templates to generate counterfactuals, which may not capture the full complexity of natural language. Additionally, the method requires researchers to pre-specify which secondary attributes are important. Future work will explore unsupervised methods to discover these attributes and the use of generative models to create more natural counterfactual data.
Summary updated