Constructing Holistic Measures for Social Biases in Masked Language Models

Yang Liu, Yuexian Hou

TL;DR

This paper introduces two novel metrics, KLDivS and JSDivS, to more holistically evaluate social biases in Masked Language Models (MLMs). By treating the log-likelihoods of stereotype and anti-stereotype sentences as Gaussian distributions, these metrics offer greater stability and interpretability compared to existing methods that rely solely on log-likelihood comparisons.

What Problem Does This Paper Address?

Large language models, particularly Masked Language Models (MLMs), have become incredibly powerful for various natural language processing tasks. However, they are trained on vast amounts of text data that unfortunately contain societal biases. These biases can be amplified by MLMs and lead to harmful stereotypes. While researchers have developed methods to detect these biases, existing evaluation metrics often have limitations. Many rely on simple comparisons of log-likelihoods for stereotype vs. anti-stereotype sentences, which can be sensitive to individual data points and lack a holistic view of the model's output distribution. This can lead to metrics that are not robust or easy to interpret, making it difficult to truly understand and mitigate the biases present in these models.

Methods

The core contribution of this paper is the proposal of two new bias evaluation metrics: the Kullback-Leibler Divergence Score (KLDivS) and the Jensen-Shannon Divergence Score (JSDivS). Instead of just looking at individual log-likelihood values, the authors treat the log-likelihoods of stereotype bias samples and anti-stereotype bias samples as probability distributions, specifically Gaussian distributions.

  • KLDivS: This metric uses the Kullback-Leibler (KL) divergence to measure the asymmetric distance between the stereotype bias distribution and the anti-stereotype bias distribution. To ensure comparability and a baseline for no bias, the score is adjusted to be at least 50, with higher values indicating less bias.
  • JSDivS: This metric utilizes the Jensen-Shannon (JS) divergence, a symmetric variant of KL divergence, to measure the distance between the two distributions. The JS divergence ranges from 0 to 1, where 0 indicates identical distributions (no bias). The authors scale this to a 0-100 range, where 100 signifies no bias.

The key innovation here is the shift from point estimates (single log-likelihoods) to distributional analysis, aiming to capture the overall behavior of the MLM across a set of biased and unbiased examples. This approach is designed to be less sensitive to individual "pitfall" samples that might not accurately reflect the model's general bias.

Datasets

The authors evaluated their proposed metrics on two publicly available datasets:

  • StereoSet (SS): This dataset contains sentence pairs designed to test for stereotype bias across four types: gender, profession, race, and religion.
  • CrowS-Pairs (CP): This dataset also features sentence pairs, but it covers a broader range of bias types, including race, gender, sexual orientation, religion, age, nationality, disability, physical appearance, and socioeconomic status.

The paper notes that for StereoSet, they used the development set as the test set was not publicly available. They also focused on the "Intrasentence" part of StereoSet.

Key Insights and Ideas

  • Beyond Log-Likelihood Ratios: The paper argues that simply comparing log-likelihoods of stereotype vs. anti-stereotype sentences is insufficient. By modeling these as distributions, we can better understand the spread and central tendency of the model's predictions, leading to a more nuanced evaluation.
  • Robustness to Pitfalls: Treating log-likelihoods as distributions makes the evaluation metrics more robust to individual "bad" or misleading data points within the datasets. This is crucial because even well-constructed datasets can contain problematic examples.
  • Interpretability: The authors emphasize that their metrics, particularly JSDivS, offer better interpretability. A score closer to 100 (for JSDivS) directly translates to less bias, making it easier to communicate findings.
  • Holistic View: The proposed metrics aim to provide a "holistic" view of bias by considering the entire output distribution rather than just a single point.

Key Results

The experimental results on StereoSet and CrowS-Pairs demonstrate that:

  • KLDivS and JSDivS are more stable: When comparing the performance of models on the full datasets versus 80% sampled subsets, KLDivS and JSDivS showed the smallest deviations (measured by Absolute Total Deviation - ATD), indicating greater stability.
  • Comparable performance to baselines: The proposed metrics generally align with existing metrics like StereS, CrowS, and AUL in identifying which models are more or less biased. For instance, BERT consistently showed less bias across most metrics and datasets.
  • JSDivS shows negative correlation with others: JSDivS is negatively correlated with other metrics because a higher JSDivS score indicates less bias, whereas higher scores for other metrics often indicate more bias. This highlights the distinct interpretation of JSDivS.
  • Dataset construction matters: The paper points out that differences in how sentence pairs are constructed in StereoSet and CrowS-Pairs can lead to variations in bias scores, underscoring the importance of careful dataset design.

Limitations and Future Work

  • Dataset Pitfalls: The authors acknowledge that even the datasets used (StereoSet and CrowS-Pairs) have inherent pitfalls, which can affect evaluation results. They note that their metrics are more stable despite these pitfalls.
  • Evaluation Error: The paper mentions an "evaluation error" for the AUL metric on the SS dataset with RoBERTa, where the result on a sampled subset differed significantly from the original dataset. This highlights the importance of stability.
  • Future Work: The authors suggest that their perspective opens new avenues for future work in measuring and debiasing social biases in MLMs. The proposed metrics provide a more robust foundation for these efforts.

Summary updated

Abstract

Masked Language Models (MLMs) have been successful in many natural language processing tasks. However, real-world stereotype biases are likely to be reflected in MLMs due to their learning from large text corpora. Most of the evaluation metrics proposed in the past adopt different masking strategies, designed with the log-likelihood of MLMs. They lack holistic considerations such as variance for stereotype bias and anti-stereotype bias samples. In this paper, the log-likelihoods of stereotype bias and anti-stereotype bias samples output by MLMs are considered Gaussian distributions. Two evaluation metrics, Kullback Leibler Divergence Score (KLDivS) and Jensen Shannon Divergence Score (JSDivS) are proposed to evaluate social biases in MLMs The experimental results on the public datasets StereoSet and CrowS-Pairs demonstrate that KLDivS and JSDivS are more stable and interpretable compared to the metrics proposed in the past.
BibTeX
@misc{liu2023constructing,
  author = {Yang Liu and Yuexian Hou},
  title = {{Constructing Holistic Measures for Social Biases in Masked Language Models}},
  year = {2023},
  eprint = {2305.07795},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.07795},
}
Figure