Adversarial Examples Generation for Reducing Implicit Gender Bias in Pre-trained Models

Wenqian Ye, Fei Xu, Yaojia Huang, Cassie Huang, Ji AarXiv 2021 · arXiv

Abstract

Pre-trained language models often harbor implicit biases that are difficult to detect using standard word-level metrics. This paper introduces an automated pipeline to generate sentence-level adversarial examples that expose structural gender bias. By filtering large text corpora for specific syntactic structures and masking gendered pronouns, the authors create a targeted dataset revealing that models like BERT lean heavily toward male pronoun predictions in gender-neutral contexts.

Introduction: Beyond Word-Level Bias

While the NLP community has made significant strides in identifying explicit gender bias—such as the stereotypical associations found in word embeddings (e.g., "man is to programmer as woman is to homemaker")—implicit, sentence-level biases remain under-explored. These biases can be categorized into two types:

  1. Contextual bias: Requires world knowledge to identify (e.g., "Presidential candidates need their wives to support them").
  2. Structural bias: Occurs when syntax enforces a gender assumption on a neutral subject (e.g., "Someone winds up his right arm").

Existing benchmarks often lack the scale or automation needed to probe these structural biases effectively. This research addresses this gap by proposing a method to automatically mine and generate sentence-level bias examples from existing corpora.

Methodology: Automated Bias Mining

The authors propose a two-stage pipeline focusing specifically on structural bias.

1. The Filter Stage

The goal is to isolate sentences where a gendered pronoun refers to a gender-neutral entity. The process involves starting with a large corpus (SWAG) and filtering out sentences with multiple sex indicators to ensure the model cannot rely on explicit gendered cues. The pipeline uses coreference resolution to find pairs where a gendered pronoun refers back to a neutral subject or object, followed by noise elimination and simplification steps.

The automated pipeline for filtering sentences to isolate structural gender bias.

2. Bias Computation

Once filtered, the gendered pronouns are masked (e.g., "The dancer shakes [MASK] hips"). These masked sentences are fed into a Masked Language Model (MLM). A bias score is calculated based on the relative probability of predicting a male pronoun $P(s|m)$ versus a female pronoun $P(s|f)$:

$$ Bias Score = \frac{P(s|m)}{P(s|m) + P(s|f)} $$

A score significantly deviating from 0.5 indicates a gender bias for that specific sentence structure.

Datasets and Benchmarking

The authors utilized the SWAG (Situations With Adversarial Generations) dataset, originally designed for grounded commonsense inference. From an initial pool of over 73,000 sentences, the filtering pipeline produced a refined benchmark of 663 sentences specifically tailored to test structural gender bias. Human evaluation confirmed that 90.79% of these generated examples were valid instances of structural bias.

Results: Pervasive Male Skew

Evaluation of the generated benchmark revealed a significant male bias across different BERT variants. The distribution of bias scores shows a clear shift toward the male end of the spectrum.

Distribution of bias scores for BERT-uncased and BERT-cased models, showing a clear male-skewed distribution.
  • BERT-uncased: 542 out of 663 sentences were male-biased, with an average bias score of 0.556.
  • BERT-cased: 575 sentences were male-biased, with an average score of 0.550.

A notable case study involved the sentence: "Someone winds up [MASK] right arm and knocks the fighter down...". While BERT-uncased showed a moderate preference for 'his' (14.2%) over 'her' (11.3%), DistilBERT exhibited a massive skew, predicting 'his' with 54.2% probability compared to just 3.05% for 'her'. This suggests that model compression techniques might inadvertently amplify implicit biases.

Discussion and Future Work

The findings demonstrate that syntax often hides deep-seated assumptions. By moving to the sentence level, the authors show that models bake in subtle gendered expectations even in neutral contexts.

While the current pipeline is highly effective for structural bias, it is less suited for contextual bias, which requires broader world knowledge. Future work will involve testing this methodology on a wider range of transformer architectures (such as ALBERT and RoBERTa) and utilizing these adversarial examples for active debiasing through data augmentation and adversarial training.

Summary updated

Abstract

Over the last few years, Contextualized Pre-trained Neural Language Models, such as BERT, GPT, have shown significant gains in various NLP tasks. To enhance the robustness of existing pre-trained models, one way is adversarial examples generation and evaluation for conducting data augmentation or adversarial learning. In the meanwhile, gender bias embedded in the models seems to be a serious problem in practical applications. Many researches have covered the gender bias produced by word-level information(e.g. gender-stereotypical occupations), while few researchers have investigated the sentence-level cases and implicit cases. In this paper, we proposed a method to automatically generate implicit gender bias samples at sentence-level and a metric to measure gender bias. Samples generated by our method will be evaluated in terms of accuracy. The metric will be used to guide the generation of examples from Pre-trained models. Therefore, those examples could be used to impose attacks on Pre-trained Models. Finally, we discussed the evaluation efficacy of our generated examples on reducing gender bias for future research.
BibTeX
@misc{ye2021adversarial,
  author = {Wenqian Ye and Fei Xu and Yaojia Huang and Cassie Huang and Ji A},
  title = {{Adversarial Examples Generation for Reducing Implicit Gender Bias in Pre-trained Models}},
  year = {2021},
  eprint = {2110.01094},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.01094},
}
Figure