An Empirical Study on Pseudo-log-likelihood Bias Measures for Masked Language Models Using Paraphrased Sentences

Bum Chul Kwon, Nandana MihindukulasooriyaNAACL 2022

TL;DR

This paper investigates the reliability of a common method for detecting social biases in language models, which relies on comparing the likelihood of stereotypical vs. non-stereotypical sentences. They found that this method is surprisingly sensitive to the specific wording of sentences, rather than their underlying meaning, suggesting a potential flaw in how we currently measure bias in these models.

What Problem Does This Paper Address?

Large language models (LLMs) like BERT and GPT have become incredibly powerful, but there's a growing concern that they inherit and even amplify social biases present in their training data. These biases can manifest as harmful stereotypes about various social groups (based on race, gender, religion, etc.). Researchers have developed methods to detect these biases, often by comparing how likely a model is to generate or complete sentences that reflect stereotypes versus those that don't. One popular technique uses "pseudo-log-likelihood scoring" on masked language models (MLMs) to quantify this preference. However, it's unclear if these measures truly capture the model's underlying beliefs or if they are simply picking up on superficial linguistic cues. This paper aims to test the robustness of these bias detection methods by seeing if they produce consistent results when the wording of a sentence changes, but its meaning remains the same.

Methods

The core of this study involves a clever experimental design: taking existing bias-measuring datasets and systematically altering the wording of sentences within them. Specifically, the researchers used the CrowS-Pairs dataset, which contains pairs of sentences designed to elicit stereotypical responses. For each pair, they manually created paraphrased versions of the sentences, ensuring the meaning stayed the same but the specific words changed. They then fed these original and paraphrased sentences into several popular MLMs (BERT, RoBERTa, ALBERT, DistilBERT, and MPNet) and calculated the pseudo-log-likelihood difference (MDIFF) between the stereotypical and non-stereotypical versions. This difference indicates which sentence the model "prefers." They also derived a binary measure (MBIAS) to classify whether the model leans towards the stereotype. The key metric for their analysis is "agreement" (MAGREEMENT), which checks if the MBIAS is consistent between the original and paraphrased sentence pairs.

Datasets

The primary dataset used in this study is a subset of the CrowS-Pairs dataset (Nangia et al., 2020). This dataset consists of contrastive sentence pairs designed to measure social biases in MLMs. The authors selected 100 examples from CrowS-Pairs and manually created 3 to 5 paraphrased versions for each, resulting in a total of 383 new samples. These paraphrased sentences were semantically identical to the originals but used different wording.

Key Insights and Ideas

  • Wording Over Meaning: The most striking finding is that the bias detection method is highly sensitive to the specific words used, rather than the overall meaning of the sentence. Even minor changes in phrasing can flip the model's perceived bias.
  • Fragility of Current Measures: This sensitivity suggests that current pseudo-log-likelihood based bias measures might be too fragile to reliably indicate a model's true social biases. They might be capturing superficial linguistic patterns rather than deeper stereotypical associations.
  • Model Variability: Different MLMs exhibit varying degrees of agreement when faced with paraphrased sentences, indicating that some models are more susceptible to these wording-based fluctuations than others.
  • Small Differences Matter: The log-likelihood differences (MDIFF) between sentence pairs are often very small (within -.25 to .25). This means that even slight changes in wording can push the MDIFF across the zero threshold, altering the perceived bias.

Key Results

The study found that the proportion of agreement (MPER_AGREE) between the original and paraphrased sentence pairs was often less than 80% across the five tested MLMs. For instance, DistilBERT Base showed 75% agreement, while MPNetBase only achieved 57%. This indicates that in a significant portion of cases, paraphrasing the sentences led to a different conclusion about the model's bias. The results also showed that the MDIFF values were frequently very close to zero, making them susceptible to changes caused by word choice.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Limited Dataset Size: The experiment used only a small subset of the CrowS-Pairs dataset. Future work should extend the analysis to the entire dataset.
  • Manual Paraphrasing: The paraphrases were created manually. Automating this process would allow for larger-scale experiments.
  • Single Bias Measure: The study focused on one specific bias measure (pseudo-log-likelihood). Future research should investigate the robustness of other bias measurement techniques.
  • Statistical Significance: The authors plan to test the statistical significance of their findings on log-likelihood based measures.
  • Improving Measures: The paper suggests exploring alternative ways to measure bias, perhaps by considering multiple thresholds for likelihood differences or reporting the magnitude of these differences, rather than just a binary classification. They also propose testing the robustness of bias measures through bootstrapping or sentence perturbation.

Summary updated

BibTeX
@inproceedings{kwon2022empirical,
  author = {Bum Chul Kwon and Nandana Mihindukulasooriya},
  title = {{An Empirical Study on Pseudo-log-likelihood Bias Measures for Masked Language Models Using Paraphrased Sentences}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure