Abstract
Pre-trained language models often harbor implicit biases that are difficult to detect using standard word-level metrics. This paper introduces an automated pipeline to generate sentence-level adversarial examples that expose structural gender bias. By filtering large text corpora for specific syntactic structures and masking gendered pronouns, the authors create a targeted dataset revealing that models like BERT lean heavily toward male pronoun predictions in gender-neutral contexts.
Introduction: Beyond Word-Level Bias
While the NLP community has made significant strides in identifying explicit gender bias—such as the stereotypical associations found in word embeddings (e.g., "man is to programmer as woman is to homemaker")—implicit, sentence-level biases remain under-explored. These biases can be categorized into two types:
- Contextual bias: Requires world knowledge to identify (e.g., "Presidential candidates need their wives to support them").
- Structural bias: Occurs when syntax enforces a gender assumption on a neutral subject (e.g., "Someone winds up his right arm").
Existing benchmarks often lack the scale or automation needed to probe these structural biases effectively. This research addresses this gap by proposing a method to automatically mine and generate sentence-level bias examples from existing corpora.
Methodology: Automated Bias Mining
The authors propose a two-stage pipeline focusing specifically on structural bias.
1. The Filter Stage
The goal is to isolate sentences where a gendered pronoun refers to a gender-neutral entity. The process involves starting with a large corpus (SWAG) and filtering out sentences with multiple sex indicators to ensure the model cannot rely on explicit gendered cues. The pipeline uses coreference resolution to find pairs where a gendered pronoun refers back to a neutral subject or object, followed by noise elimination and simplification steps.
2. Bias Computation
Once filtered, the gendered pronouns are masked (e.g., "The dancer shakes [MASK] hips"). These masked sentences are fed into a Masked Language Model (MLM). A bias score is calculated based on the relative probability of predicting a male pronoun $P(s|m)$ versus a female pronoun $P(s|f)$:
$$ Bias Score = \frac{P(s|m)}{P(s|m) + P(s|f)} $$
A score significantly deviating from 0.5 indicates a gender bias for that specific sentence structure.
Datasets and Benchmarking
The authors utilized the SWAG (Situations With Adversarial Generations) dataset, originally designed for grounded commonsense inference. From an initial pool of over 73,000 sentences, the filtering pipeline produced a refined benchmark of 663 sentences specifically tailored to test structural gender bias. Human evaluation confirmed that 90.79% of these generated examples were valid instances of structural bias.
Results: Pervasive Male Skew
Evaluation of the generated benchmark revealed a significant male bias across different BERT variants. The distribution of bias scores shows a clear shift toward the male end of the spectrum.
- BERT-uncased: 542 out of 663 sentences were male-biased, with an average bias score of 0.556.
- BERT-cased: 575 sentences were male-biased, with an average score of 0.550.
A notable case study involved the sentence: "Someone winds up [MASK] right arm and knocks the fighter down...". While BERT-uncased showed a moderate preference for 'his' (14.2%) over 'her' (11.3%), DistilBERT exhibited a massive skew, predicting 'his' with 54.2% probability compared to just 3.05% for 'her'. This suggests that model compression techniques might inadvertently amplify implicit biases.
Discussion and Future Work
The findings demonstrate that syntax often hides deep-seated assumptions. By moving to the sentence level, the authors show that models bake in subtle gendered expectations even in neutral contexts.
While the current pipeline is highly effective for structural bias, it is less suited for contextual bias, which requires broader world knowledge. Future work will involve testing this methodology on a wider range of transformer architectures (such as ALBERT and RoBERTa) and utilizing these adversarial examples for active debiasing through data augmentation and adversarial training.
Summary updated