Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models

Bum Chul Kwon, Nandana MihindukulasooriyaACL 2023

TL;DR

This paper introduces Finspector, a novel human-centered visual inspection tool designed to help researchers and practitioners explore and compare biases within large language models (LLMs). By leveraging log-likelihood scores and interactive visualizations, Finspector aims to make the complex task of bias detection more accessible and intuitive, ultimately contributing to the fairer deployment of LLMs.

What Problem Does This Paper Address?

The increasing prevalence and power of pre-trained foundation models (LLMs) in natural language processing (NLP) tasks have brought to light a critical concern: these models often inherit and amplify societal biases present in their training data. These biases can lead to discriminatory outcomes and reinforce harmful stereotypes, making it crucial to develop methods for their detection and mitigation. While various benchmarks and tools exist for identifying bias, many are designed for specific model types (e.g., image classification) or lack the interactive, human-centered approach needed to effectively explore the nuanced biases within LLMs. This paper addresses the need for a tool that allows users to visually inspect and compare biases across different foundation models in an intuitive and systematic way.

Methods

Finspector is a Python package designed for use in interactive computing environments like Jupyter notebooks. Its core functionality revolves around a human-centered visual inspection approach. The tool utilizes pseudo-log-likelihood scores, calculated by masking tokens in sentences and averaging their probabilities, as a measure of how likely a language model is to produce a given sentence.

The system presents three main interactive views:

  • Distribution of Log-Likelihoods: This view displays the distribution of these scores across different foundation models, allowing users to compare their performance. It supports interactive filtering and the creation of parallel coordinates to visualize score differences between models.
  • Table of Sentences: A detailed view of the input sentences, which is tightly integrated with the other views. Users can see sentence details, bias categories, and log probability scores, with highlighting and filtering synchronized across all views.
  • Sentence Embeddings: This view uses scatterplots to visualize sentence similarities and differences based on their embedding vectors. Users can select sentences via lassoing, which then highlights them in the other views.

A key aspect of Finspector is its interactive customization panel, which allows users to highlight sentences based on bias categories and split distributions by selected variables, facilitating a deeper exploration of model behavior.

Datasets

The paper mentions the use of the CrowS-Pairs dataset for their use case. This dataset is described as containing paired samples and bias category labels, similar to other benchmarks like StereoSet, WinoGender, and WinoBias, which are also cited in the background section. The authors generated 10 paraphrased sentences per given sentence from CrowS-Pairs using a method called Pegasuss to increase the sample size for their analysis.

Key Insights and Ideas

  • Human-in-the-loop is crucial: The authors emphasize the need for a systematic, human-in-the-loop approach to inspect fairness and biases, rather than relying solely on automated metrics.
  • Visual analytics for LLM bias: Finspector demonstrates how interactive visualizations can significantly aid in understanding the complex inner workings of LLMs and identifying potential biases.
  • Log-likelihood as a bias indicator: The paper reinforces the utility of pseudo-log-likelihood scores as a measure for detecting model preferences for stereotypical versus non-stereotypical sentences.
  • Cross-model comparison is vital: By allowing users to compare multiple foundation models side-by-side, Finspector highlights how different models exhibit varying degrees and types of bias.
  • Interactive exploration reveals nuances: The ability to filter, highlight, and drill down into specific sentences and categories allows for the discovery of subtle bias patterns that might be missed by static analyses.

Key Results

The paper presents a use case demonstrating Finspector's capabilities. When analyzing the CrowS-Pairs dataset with BERT, ALBERT, and RoBERTa:

  • Initially, the three models showed similar distributions of log probabilities, with ALBERT's scores being slightly lower, indicating a potential tendency to assign lower probabilities to certain sentences.
  • Filtering on ALBERT's scores revealed that sentences with lower probabilities on ALBERT tended to have higher values on BERT or RoBERTa, suggesting differing sensitivities to bias across models.
  • The analysis of the disability category showed that stereotypical sentences appeared more likely than base sentences for ALBERT.
  • Customized sentences demonstrated that gender stereotypes (e.g., "Housekeeping and childcare are women's responsibility") often received higher log probabilities than their counterparts (e.g., "Housekeeping and childcare are men's responsibility") across all three models.

Limitations and Future Work

Limitations:

  • The authors acknowledge that there is "no guarantee to discover all biases or fairness issues by using the tool."
  • The current implementation primarily focuses on encoder-only models (BERT, RoBERTa, ALBERT) that use masked language modeling objectives.

Future Work:

  • Expand model support: Incorporate other families of language models, such as decoder-only (e.g., GPT) and encoder-decoder (e.g., BART, T5) models, into Finspector. This would involve adapting or developing log-likelihood-equivalent measures for these architectures.
  • Investigate other bias measures: Explore and integrate alternative or complementary bias measures beyond pseudo-log-likelihood scoring.
  • Develop more advanced visualizations: Investigate further visual analytics approaches for inspecting fairness and biases in models trained with various objectives and architectures.
  • Enhance robustness evaluation: Develop methods to improve the robustness of pseudo-log-likelihood scoring as a bias measure, especially for foundation models adapted to downstream tasks.
  • Create benchmark datasets: Curate benchmark datasets with stereotype sentence pairs in a systematic manner to facilitate more standardized evaluations.
  • Interdisciplinary research: Foster collaboration between AI, visualization, and other fields to build more comprehensive systems for evaluating the impact of biases in foundation models.

Summary updated

BibTeX
@inproceedings{kwon2023finspector,
  author = {Bum Chul Kwon and Nandana Mihindukulasooriya},
  title = {{Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure