Evaluating the Efficacy of Prompting Techniques for Debiasing Language Model Outputs (Student Abstract)

Shaz Furniturewala, Surgan Jandial, Abhinav Java, Simra Shahid, Pragyan Banerjee, Balaji Krishnamurthy, Sumit Bhatia, Kokil JaidkaAAAI 2024 · OpenReview

TL;DR

This paper investigates how different prompting strategies can reduce biases in Large Language Models (LLMs). They found that reasoning-focused prompts, while effective at debiasing, sometimes sacrifice fluency, suggesting a trade-off between fairness and naturalness in LLM outputs.

What Problem Does This Paper Address?

Large Language Models (LLMs), trained on vast amounts of internet data, often inherit and perpetuate societal biases. This can lead to unfair or discriminatory outputs in various applications. While methods like data augmentation exist to combat this, they are computationally expensive for massive models. Prompt engineering has emerged as a flexible way to steer LLM behavior, but its potential for debiasing has been underexplored. This paper aims to systematically evaluate various prompting techniques for their effectiveness in reducing bias in LLM-generated text.

Methods

The researchers adopted a zero-shot prompting approach, meaning they didn't fine-tune the LLMs. They tested several prompting strategies:

  • Instruction: A simple directive to generate unbiased text.
  • Role/Persona: Assigning the LLM the role of an unbiased individual.
  • Zero-Shot CoT (Chain-of-Thought): Encouraging step-by-step reasoning to achieve fairness.
  • Self Debias: A post-hoc method that uses prefixes to generate biased text and then penalizes words with high probabilities in that biased distribution.
  • Implicative Introspection: A novel iterative approach where the LLM's output is fed back with a directive asking about its implications, leading to a refined, unbiased output.

Datasets

The primary dataset used for evaluation was StereoSet. StereoSet is a natural language dataset designed to measure stereotypical bias across four categories: gender, race, religion, and profession. It presents sentences with a blank to be filled and offers three options: a stereotype, an anti-stereotype, and an unrelated word. The dataset allows for measuring how likely an LLM is to choose the stereotypical option.

Key Insights and Ideas

  • Reasoning vs. Fluency Trade-off: Prompts that encourage reasoning (like Zero-Shot CoT and Implicative Introspection) are generally more effective at reducing bias. However, they sometimes lead to a decrease in the fluency of the generated text, as indicated by lower LM (Language Modeling) scores. This suggests a potential trade-off between achieving fairness and maintaining natural language generation quality.
  • Uniformity of Debiasing: The debiasing effect of the tested prompts was observed to be relatively uniform across different types of bias (gender, race, religion, profession).
  • Model Scale Matters: The largest model tested, LLaMA 2 (13B), showed the greatest overall improvement in debiasing, particularly with the self-debiasing approach.

Key Results

The paper presents results in two tables, showing StereoSet (SS) scores and LM/ICAT scores for various models and prompting techniques.

  • StereoSet Scores (Table 1): Reasoning-based prompts like "Zero-Shot CoT" and "Implication" generally achieved lower (better) SS scores, indicating less stereotypical output. For example, on GPT-2 Large, "Zero-Shot CoT" achieved an SS score of 57.47, compared to the base model's 60.42.
  • LM and ICAT Scores (Table 2): While reasoning prompts reduced bias, they often resulted in lower LM scores (indicating less fluent or natural text). For instance, the "Implication" prompt on LLaMA 2 had an ICAT score of 66.42, but its LM score was 83.00, lower than the base model's 63.62. The "Instruction" prompt, while less effective at debiasing, often maintained higher LM scores.
  • LLaMA 2 Performance: The LLaMA 2 (13B) model demonstrated the most significant improvements in debiasing across various prompts, especially with the "Self Debias" approach.

Limitations and Future Work

Limitations: * The study focused on a specific debiasing benchmark (StereoSet), and the findings might not generalize to all types of bias or all LLM applications. * The trade-off between debiasing efficacy and text fluency is a significant limitation that needs further investigation.

Future Work: * Expand the evaluation to a wider range of datasets, such as BOLD and CrowSPairs, and include more LLMs. * Explore more elaborate research designs that combine instructive and reasoning approaches in prompting to potentially achieve both high fairness and fluency. * Develop a more general structured prompting framework for debiasing that can generalize across different texts and LLMs.

Summary updated

Abstract

Achieving fairness in Large Language Models (LLMs) continues to pose a persistent challenge, as these models are prone to inheriting biases from their training data, which can subsequently impact their performance in various applications. There is a need to systematically explore whether structured prompting techniques can offer opportunities for debiased text generation by LLMs. In this work, we designed an evaluative framework to test the efficacy of different prompting techniques for debiasing text along different dimensions. We aim to devise a general structured prompting approach to achieve fairness that generalizes well to different texts and LLMs.
BibTeX
@inproceedings{furniturewala2024evaluating,
  author = {Shaz Furniturewala and Surgan Jandial and Abhinav Java and Simra Shahid and Pragyan Banerjee and Balaji Krishnamurthy and Sumit Bhatia and Kokil Jaidka},
  title = {{Evaluating the Efficacy of Prompting Techniques for Debiasing Language Model Outputs (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=4AzRmVe0Zt},
}
Figure