Abstract
Despite remarkable performances in various applications, machine learning (ML) models can potentially discriminate, leading to biased decision-making with negative societal impacts. While various debiasing methods exist, the explicit influence of attention mechanisms—a core component of modern NLP—on model fairness remains under-explored. This paper investigates how different attention mechanisms affect the fairness of RNN-based models (BiGRU and BiLSTM) across three benchmark datasets: Jigsaw, HateXplain, and HSOL. Our results indicate that most attention mechanisms improve fairness regarding religious and gender-sensitive groups, albeit often at the cost of accuracy.
Introduction
The proliferation of Large Language Models (LLMs) has brought the issue of algorithmic bias to the forefront. Models often inherit and amplify societal prejudices present in their training data, such as gendered professional stereotypes. While attention mechanisms are celebrated for improving model performance by allowing the network to focus on relevant parts of the input, their role in shaping fairness is a critical area of inquiry. We aim to determine if attention can serve as an implicit debiasing tool in the context of offensive language classification.
Methodology
We conducted a comparative analysis using two standard architectures: the Bidirectional Gated Recurrent Unit (BiGRU) and the Bidirectional Long Short-Term Memory (BiLSTM). Each model was evaluated in four configurations:
- Baseline: No attention mechanism.
- Additive Attention: Utilizes a learned weight matrix to calculate token importance.
- Dot Product Attention: Determines importance based on the similarity between hidden states.
- Self-Attention: Captures dependencies between tokens within the same sequence.
Fairness was measured using Equalized Odds (EO), which quantifies the disparity in True Positive and False Positive rates across different demographic groups. A lower EO score signifies a fairer model. Performance was tracked using the Balanced F1 score.
Experimental Setup and Datasets
The study utilized three datasets focused on toxicity and hate speech:
- Jigsaw: Online news comments from a Kaggle competition.
- HateXplain: Twitter-sourced data including human-provided "explanations" for labels.
- HSOL (Hate Speech Offensive Language): A collection of tweets categorized via a hate speech lexicon.
Test sets were specifically curated by filtering for keywords related to religious and gender-sensitive identities to evaluate model behavior toward protected groups.
Key Results and the Fairness-Accuracy Trade-off
Our findings reveal a consistent trend: the inclusion of attention mechanisms generally enhances model fairness. In the Jigsaw dataset, for instance, the BiLSTM with Self-Attention achieved the highest fairness for religious groups (EO: 0.0020).
However, this improvement often necessitates a "fairness-accuracy trade-off." As seen in Figure 1, models that minimize EO (moving toward the left of the x-axis) frequently experience a decrease in their Balanced F1 scores. For example, in the Jigsaw experiments, Additive Attention yielded superior fairness but resulted in the most significant accuracy loss.
Discussion: The "Black" Keyword Trap
While attention can mitigate bias, it is not a panacea. Visualization of attention weights through heatmaps provides insight into how these models process sensitive information.
As illustrated in Figure 6, models—even those with improved fairness metrics—can still fixate on sensitive tokens like "black," "muslim," or "bitch." If the training data contains a disproportionate number of toxic labels for sentences containing these words, the attention mechanism may inadvertently learn to treat these identity markers as proxies for toxicity. This highlights the importance of data quality and the limitations of architectural interventions alone.
Conclusion
This research demonstrates that attention mechanisms can significantly influence the fairness of NLP models. While they often act as implicit debiasing tools, the resulting "accuracy tax" and the potential for models to fixate on biased proxies remain challenges. Future work should explore compound attention modules and the use of specialized loss functions, such as Focal or Dice loss, to better handle the data imbalances that often drive algorithmic bias.
Summary updated