Bias Silhouette Analysis: Towards Assessing the Quality of Bias Metrics for Word Embedding Models

Maximilian Spliethöver, Henning WachsmuthIJCAI 2021 · DOI

TL;DR

This paper introduces Bias Silhouette Analysis (BSA), a novel method to rigorously evaluate the quality of bias metrics used for word embeddings. BSA quantifies how robust and accurate these metrics are, revealing that current metrics and word lists often struggle to reliably distinguish between biased and unbiased models.

What Problem Does This Paper Address?

Word embeddings, which represent words as numerical vectors, can inadvertently learn and even amplify societal biases present in their training data. This is a significant concern as these embeddings are increasingly used in real-world applications, potentially leading to discriminatory outcomes. Researchers have developed metrics like WEAT, ECT, and RNSB to quantify these biases, often relying on predefined lists of words representing social groups and bias-conveying concepts. However, the reliability and accuracy of these metrics, and crucially, the quality of the word lists they employ, have not been systematically assessed. This paper aims to fill that gap by providing a framework to evaluate how well these metrics and their associated word lists actually measure social bias.

Methods

The core contribution is the Bias Silhouette Analysis (BSA). BSA works by systematically evaluating a bias metric on progressively larger subsets of a given word list. This is done using two reference word embedding models: one assumed to be biased (GloVe) and one assumed to be unbiased (NBatch).

  1. Bias Silhouette Computation: For each model and metric, BSA generates random subsets of the word lists, increasing in size. For each subset, the bias metric is applied to compute a bias value. This process is repeated many times to create a sequence of bias values for each subset size. These sequences are then used to define a "bias silhouette" for each model, which is essentially a plot showing the range (min/max) of bias values observed across different subset sizes.
  2. Robustness Score: The robustness of a word list is assessed by how much the bias metric's output varies as the subset size changes. A more robust list will produce stable bias values, resulting in a narrow silhouette. This is quantified by a "robustness score" that measures the area of the silhouette relative to the maximum possible area.
  3. Accuracy Score: The accuracy of a bias metric is evaluated by how well it distinguishes between the biased and unbiased reference models. An accurate metric should assign a low bias value to the unbiased model and a high value to the biased model. This is captured by the "accuracy score," which measures the area between the mean bias values of the two silhouettes.

Datasets

The paper uses existing word lists for three types of social bias:

  • Ethnicity: African-American and European-American names.
  • Gender: Male and female terms, first names, and professions.
  • Religion: Christian and Islamic terms.

These lists are combined from prior research (Caliskan et al., Garg et al., Bolukbasi et al., Manzini et al.).

The evaluation is performed on two pre-trained word embedding models:

  • Biased Reference Model: GloVe CommonCrawl.
  • Unbiased Reference Model: ConceptNet Numberbatch (NBatch), which was explicitly debiased.

Key Insights and Ideas

  • The quality of bias metrics is highly variable: Current metrics and word lists are not equally effective across different types of bias or embedding models.
  • Word list robustness matters: The choice of words in a list significantly impacts the stability of bias measurements. Some lists are highly sensitive to the inclusion or exclusion of specific words.
  • Accuracy isn't guaranteed: Even with established metrics, it's challenging to reliably differentiate between biased and unbiased models, suggesting that either the models are not as extreme as assumed, or the metrics are not sensitive enough.
  • Longer word lists aren't always better: While intuition might suggest more words lead to better bias detection, the study found that shorter lists can sometimes suffice, and larger lists don't always guarantee higher robustness.
  • Failures to embed words are critical: When words from the lists cannot be found in the embedding model (out-of-vocabulary), it can significantly affect the bias measurements, especially for smaller models.

Key Results

  • Robustness: The ECT metric generally showed the most robust performance across different word lists. WEAT's robustness scores varied significantly, and RNSB showed low robustness for ethnicity bias on the GloVe model.
  • Accuracy: ECT was found to be the least accurate metric, with scores sometimes falling below 0.5 (indicating no clear distinction between models). RNSB generally performed better in terms of accuracy, especially for gender and religion bias. WEAT achieved the highest accuracy scores for ethnicity bias.
  • Model Dependence: Robustness scores varied depending on whether the GloVe or NBatch model was used, highlighting that metric performance can be model-specific.
  • Overall: In most cases, the evaluated metrics struggled to clearly distinguish between the biased GloVe and the less biased NBatch models, suggesting limitations in current bias quantification approaches.

Limitations and Future Work

  • Reference Model Assumption: The primary limitation is the reliance on assumed biased (GloVe) and unbiased (NBatch) models. The authors acknowledge that these might not be true extreme points of bias, and the exact level of bias in the training data is unknown. They suggest that BSA can be applied again with better-defined reference models.
  • Word List Quality: The paper uses existing word lists without a thorough examination of their completeness, correctness, or potential cultural biases. The lists are also geared towards direct bias rather than indirect bias and assume Western-centric views.
  • Embedding Algorithm: The study did not evaluate the impact of different embedding algorithms on bias metric performance.

Future work could involve constructing explicitly biased and unbiased datasets to train reference models, and further investigating the impact of word list construction and embedding algorithms on bias measurement. The authors have also published their code to facilitate further research.

Summary updated

BibTeX
@inproceedings{spliethover2021bias,
  author = {Maximilian Spliethöver and Henning Wachsmuth},
  title = {{Bias Silhouette Analysis: Towards Assessing the Quality of Bias Metrics for Word Embedding Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/77},
}
Figure