ROBBIE: Robust Bias Evaluation of Large Generative Language Models

David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, Eric Michael Smith

Abstract

As Large Language Models (LLMs) become foundational to modern software, the lack of standardized, multi-dimensional fairness benchmarks has hindered the ability to compare models and evaluate mitigation strategies. This paper introduces ROBBIE (Robust Bias Evaluation), a comprehensive framework designed to move beyond narrow, single-axis fairness tests. By evaluating five model families across 12 demographic axes using six metrics—including two newly introduced datasets—ROBBIE provides a robust methodology for measuring how social biases manifest and how effectively common mitigation strategies perform across different architectures and scales.

Introduction

Measuring fairness in LLMs has historically been a fragmented endeavor, often resembling a game of "Whac-A-Mole." Existing studies frequently focus on isolated demographic groups or specific metrics, making it nearly impossible to determine if a mitigation strategy that works for one group (e.g., gender) fails for another (e.g., race). Furthermore, general toxicity metrics often fail to distinguish between a model that is broadly offensive and one that specifically targets marginalized identities.

ROBBIE addresses these gaps by providing a unified benchmarking framework. It accounts for intersectional identities—such as the specific treatment of "Black women" rather than just "Black people" or "women"—and introduces a standardized "BiasScore" to isolate targeted bias from general model toxicity.

The ROBBIE Framework

Benchmarking and the BiasScore

The researchers evaluated five prominent model families: GPT-2, OPT, BlenderBot 3, BLOOM, and LLaMa. To provide a nuanced view of fairness, they defined the BiasScore. This metric calculates the percentage of demographic subgroups for which a model’s negative output frequency exceeds a predefined "background" threshold. This distinction is critical; it allows researchers to identify models that may have low overall toxicity but exhibit high levels of bias toward specific, vulnerable populations.

New Benchmarks for Intersectional Bias

While the study utilizes established datasets like BOLD and ToxiGen, it introduces two significant new benchmarks to the field:

  1. AdvPromptSet: A novel adversarial dataset derived from Jigsaw toxicity data. It is specifically engineered to test intersectional biases by querying prompts that contain pairs of demographic terms (e.g., Race × Gender), revealing harms that single-axis tests often miss.
  2. HolisticBiasR: An expansion of the "Regard" dataset. By injecting over 700 demographic terms from the HolisticBias dataset into standard templates, the authors created a tool to measure "regard" (respect and esteem) across a significantly wider array of identities, including disability, age, and socioeconomic status.

Mitigation Analysis

ROBBIE was used to test the efficacy of three popular de-biasing techniques across different model architectures:

  • Prompting: Utilizing hand-written or automatically revised instructions to guide the model toward less biased outputs.
  • Self-debiasing: A post-hoc technique where the model is tasked with diagnosing its own potential bias, subsequently shifting token probabilities to avoid biased generations.
  • Adversarial Triggering: Identifying specific prefix strings that can be used to optimally control and reduce biased outputs.

Key Insights and Results

The Complexity of Scaling

One of the most significant findings is that bias does not simply "scale up" with model size. While general toxicity often increases as models grow, the BiasScore—representing the targeting of specific groups—does not follow a predictable linear path. This suggests that increasing model parameters does not inherently improve or worsen fairness, necessitating specific intervention regardless of scale.

Model-Dependent Mitigation

The study found that there is no "silver bullet" for bias mitigation. The effectiveness of a technique is highly dependent on the model architecture. For instance, self-debiasing proved highly effective for GPT2-XL, reducing the mean toxicity rate from 1.66% to 0.59%. However, the same technique was significantly less effective for BlenderBot 3. Conversely, simple prompting was more effective for the larger BlenderBot (175B) model, reducing negative regard for marginalized groups by an average of 28%.

The "Default" Category Problem

Analysis of pre-training data (Common Crawl and Wikipedia) revealed a subtle but pervasive form of data bias. Female descriptors appeared more frequently than male ones, leading the authors to hypothesize that "male" is often treated as the "unmarked" or default category in text. Consequently, gender is often only specified when it deviates from this perceived default, embedding a structural bias into the model's foundational understanding of identity.

Intersectional Harms

The use of AdvPromptSet revealed that models often exhibit unique biases at the intersection of identities. For example, GPT2-XL showed significantly higher toxicity when responding to prompts involving the intersection of "transgender" and "homosexual" labels compared to either label in isolation.

Limitations and Future Directions

While ROBBIE provides a more robust framework than previous benchmarks, the authors acknowledge several limitations. The study is currently limited to English-language models and reflects US-centric social categories. Furthermore, the evaluation focused on "base" foundational models. Modern consumer-facing models often undergo Reinforcement Learning from Human Feedback (RLHF), which may alter their bias profiles in ways not captured here.

There is also a risk that some mitigation techniques may merely be "hiding" bias—making it harder to trigger—rather than removing the underlying associations learned during pre-training. Future work will focus on expanding ROBBIE to multilingual contexts and evaluating the evolution of bias after instruction tuning and RLHF.

Summary updated

Abstract

As generative large language models (LLMs) grow more performant and prevalent, we must develop comprehensive enough tools to measure and improve their fairness. Different prompt-based datasets can be used to measure social bias across multiple text domains and demographic axes, meaning that testing LLMs on more datasets can potentially help us characterize their biases more fully, and better ensure equal and equitable treatment of marginalized demographic groups. In this work, our focus is two-fold: (1) Benchmarking: a comparison of 6 different prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative LLMs. Out of those 6 metrics, AdvPromptSet and HolisticBiasR are novel datasets proposed in the paper. The comparison of those benchmarks gives us insights about the bias and toxicity of the compared models. Therefore, we explore the frequency of demographic terms in common LLM pre-training corpora and how this may relate to model biases. (2) Mitigation: we conduct a comprehensive study of how well 3 bias/toxicity mitigation techniques perform across our suite of measurements. ROBBIE aims to provide insights for practitioners while deploying a model, emphasizing the need to not only measure potential harms, but also understand how they arise by characterizing the data, mitigate harms once found, and balance any trade-offs. We open-source our analysis code in hopes of encouraging broader measurements of bias in future LLMs.
BibTeX
@inproceedings{esiobu2023robbie,
  author = {David Esiobu and Xiaoqing Tan and Saghar Hosseini and Megan Ung and Yuchen Zhang and Jude Fernandes and Jane Dwivedi-Yu and Eleonora Presani and Adina Williams and Eric Michael Smith},
  title = {{ROBBIE: Robust Bias Evaluation of Large Generative Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.230},
}
Figure