A Trip Towards Fairness: Bias and De-Biasing in Large Language Models

Leonardo Ranaldi, Elena Sofia Ruzzetti, Davide Venditti, Dario Onorati, Fabio Massimo Zanzotto

TL;DR

This paper investigates social biases in "Cheap-to-Build" Large Language Models (CtB-LLMs) like LLaMA and OPT, revealing that bias correlates more strongly with model perplexity than with parameter count. The authors demonstrate that parameter-efficient fine-tuning using LoRA and anti-stereotypical data can significantly reduce these biases without degrading the models' performance on standard downstream tasks.

What Problem Does This Paper Address?

The rise of Large Language Models (LLMs) has revolutionized AI, but their massive size makes them inaccessible to most researchers. Recently, "Cheap-to-Build" models (CtB-LLMs) like Meta’s LLaMA and Hugging Face’s BLOOM have democratized access by providing high-performance models that can be run on more modest hardware.

However, these models are often trained on massive, uncurated web crawls, leading them to inherit and amplify societal biases regarding gender, race, religion, and profession. While we know LLMs are biased, the relationship between a model's size, its language modeling ability (perplexity), and its level of bias remains contested. This paper seeks to quantify these biases in accessible models and provide a lightweight method to mitigate them.

Methods

The researchers employed a two-pronged approach: comprehensive evaluation and efficient mitigation.

  1. Evaluation: They used the StereoSet benchmark to measure bias across four domains (gender, profession, race, and religion) and the GLUE benchmark to ensure that any "de-biasing" didn't break the model's core linguistic capabilities.
  2. Debiasing via LoRA: Instead of retraining these massive models from scratch—which is computationally prohibitive—the authors used Low-Rank Adaptation (LoRA). They froze the majority of the model parameters and only trained the attention matrices (representing less than 0.5% of total parameters).
  3. Perturbation Data: The models were fine-tuned using the PANDA dataset, which consists of pairs of sentences where stereotypical demographic references are swapped (e.g., "women like shopping" becomes "men like shopping") to create anti-stereotypical training signals.

Datasets

  • StereoSet: A benchmark consisting of sentence triples (stereotypical, anti-stereotypical, and unrelated) used to calculate the Stereotype Score ($ss$) and Language Modeling Score ($lms$).
  • GLUE (General Language Understanding Evaluation): A collection of diverse natural language tasks (like sentiment analysis and logical inference) used to measure general model performance.
  • PANDA: A dataset of 98,000 human-annotated sentence pairs designed for "perturbation augmentation" to provide counter-stereotypical examples for training.
  • Pre-trained Models: The study evaluated the LLaMA (7B to 30B), OPT (350M to 6.7B), and BLOOM (560M to 7B) families.

Key Insights and Ideas

  • Perplexity vs. Parameters: A major finding is that bias does not necessarily increase just because a model has more parameters. Instead, within a model family, bias shows a strong negative correlation with perplexity. As a model becomes a better "language predictor" (lower perplexity), it tends to become more biased, likely because it has more effectively memorized the stereotypical associations present in its training data.
Figure 1: Model bias (ss) against model size (1a) and perplexity (1b). All measures have been standardized across the two different families of models. Our experiments suggest a lack of correlation between model size and bias (1a). A negative correlation can be observed (1b) across the different domains between perplexity and ss score while it is not possible to establish its statistical significance due to the limited number of models.
  • The Power of Curation: The BLOOM family was found to be significantly fairer than LLaMA or OPT. The authors attribute this to BLOOM's training data being manually curated by over 100 workshop participants specifically to minimize harmful content and bias.
  • Domain-Specific Bias: Across all models, gender and profession domains showed the highest levels of bias, suggesting these stereotypes are more deeply embedded in common web text than racial or religious ones.

Key Results

  • Effective Debiasing: For the OPT family, the proposed LoRA-based debiasing increased the average Normalized Stereotype Score ($nss$) by 4.12 points.
  • Race Domain Improvements: The most significant gains were seen in the race domain, where the $nss$ for OPT models increased by an average of 7.26 points.
  • Performance Stability: Crucially, the debiasing process did not degrade performance on downstream tasks. In fact, for LLaMA and OPT, performance on several GLUE tasks (like CoLA and MRPC) stayed stable or even improved slightly after the bias mitigation process.

Detailed scores across all domains and models are captured in the following table:

Table 3: StereoSet scores showing the reduction in bias after the proposed debiasing method.

Limitations and Future Work

The authors acknowledge several constraints:

  • Hardware Limits: Due to VRAM limitations (48GB), they were unable to test the largest versions of these models, such as LLaMA 65B.
  • Narrow Bias Definitions: The study relies on specific Western-centric definitions of stereotype and bias, which may not generalize across all cultures and languages.
  • The Fairness-Performance Trade-off: While LoRA mitigated the impact, the authors noted a general trend where highly "fair" models (like BLOOM) sometimes underperform on specific linguistic tasks compared to their more biased counterparts. Future work will focus on closing this gap and expanding the analysis to even larger model architectures.

Summary updated

Abstract

Cheap-to-Build Very Large-Language Models (CtB-LLMs) with affordable training are emerging as the next big revolution in natural language processing and understanding. These CtB-LLMs are democratizing access to trainable Very Large-Language Models (VLLMs) and, thus, may represent the building blocks of many NLP systems solving downstream tasks. Hence, a little or a large bias in CtB-LLMs may cause huge harm. In this paper, we performed a large investigation of the bias of three families of CtB-LLMs, and we showed that debiasing techniques are effective and usable. Indeed, according to current tests, the LLaMA and the OPT families have an important bias in gender, race, religion, and profession. In contrast to the analysis for other LLMs, we discovered that bias depends not on the number of parameters but on the perplexity. Finally, the debiasing of OPT using LoRA reduces bias up to 4.12 points in the normalized stereotype score.
BibTeX
@inproceedings{ranaldi2024trip,
  author = {Leonardo Ranaldi and Elena Sofia Ruzzetti and Davide Venditti and Dario Onorati and Fabio Massimo Zanzotto},
  title = {{A Trip Towards Fairness: Bias and De-Biasing in Large Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure