Abstract
Existing debiasing techniques are typically training-based or require access to the model's internals and output distributions, so they are inaccessible to end-users looking to adapt LLM outputs for their particular needs. In this study, we examine whether structured prompting techniques can offer opportunities for fair text generation. We evaluate a comprehensive end-user-focused iterative framework of debiasing that applies System 2 thinking processes for prompts to induce logical, reflective, and critical text generation, with single, multi-step, instruction, and role-based variants. By systematically evaluating many LLMs across many datasets and different prompting strategies, we show that the more complex System 2-based Implicative Prompts significantly improve over other techniques demonstrating lower mean bias in the outputs with competitive performance on the downstream tasks. Our work offers research directions for the design and the potential of end-user-focused evaluative frameworks for LLM use.
Introduction
The rapid adoption of Large Language Models (LLMs) has brought concerns regarding social bias to the forefront. Most existing debiasing techniques are "white-box" methods, requiring retraining, fine-tuning, or manipulation of internal output probabilities (logits). However, for end-users accessing powerful models like GPT-4 or Claude via APIs, these methods are inaccessible. Furthermore, retraining is computationally expensive and risks "catastrophic forgetting," where a model's general performance degrades as it becomes fairer. This study investigates an alternative: "black-box" debiasing through structured prompting, enabling users to mitigate bias without altering model weights.
Methodology: System 2 Prompting
Inspired by Daniel Kahneman’s dual-process theory, we categorize prompting strategies into three levels of "System 2" (slow, logical) complexity:
- Prefix Prompting (PP): A simple zero-shot approach that prepends a debiasing instruction (e.g., "The following text is unbiased...") or a persona (e.g., "You are an unbiased person...") to the user's input.
- Self-Refinement (SR): An iterative process where the model critiques its own initial output and rewrites it to be fairer over $k$ steps.
- Implication Prompting (IP): The most sophisticated method, involving a three-step chain: generating an initial output, articulating the underlying stereotype or bias (the "implication"), and finally rewriting the text based on that logical reflection.
We evaluated these strategies using both Instruction-based (direct commands) and Role-based (persona assignment) variants across four models: GPT-J (6B), Mistral-7B, Llama-2-13B, and MPT-7B.
Evaluation Framework
The framework was tested against several benchmarks:
- StereoSet: Evaluates bias in gender, race, religion, and profession via the Idealized Context Association Test (ICAT) score.
- Regard: Measures social perception and sentiment toward demographic groups.
- RealToxicityPrompts: Assesses the probability of generating toxic content.
- TruthfulQA & BoolQ: Ensures that debiasing does not compromise general reasoning and factuality.
Results and Analysis
Our findings indicate that structured reasoning is a powerful tool for debiasing. Implication Prompting (IP) emerged as the most effective strategy, achieving a 4.05% lower Stereotype Score and a 6.8% higher ICAT score on StereoSet compared to other methods. On the Regard metric, IP showed an average improvement of 26.85% in generating neutral or positive sentiment.
Key Insights
- The Power of Reflection: Forcing a model to articulate a stereotype before correcting it (IP) is significantly more effective than simple instructions. This logical bridge makes the debiasing process more robust.
- Personas vs. Instructions: Role-based prompts ("You are a fair-minded person") consistently outperformed instruction-based prompts, likely by activating a more cohesive set of behaviors within the model's latent space.
- Efficiency of Iteration: In Self-Refinement, the most significant fairness gains occur at the first step ($k=1$), with diminishing returns in subsequent iterations.
- No Performance Trade-off: Unlike training-based methods, these prompting strategies maintained or slightly improved performance on downstream tasks like TruthfulQA, avoiding the "fairness tax."
Conclusion
Structured "System 2" prompting, particularly Implication Prompting, offers a viable path for end-user debiasing of black-box LLMs. While limited to models capable of following complex instructions, this approach demonstrates that logical reflection can mitigate social bias without sacrificing performance. Future work will explore more advanced reasoning structures like Tree-of-Thought and address more nuanced, intersectional biases.
Summary updated