Abstract
When trained on large, unfiltered crawls from the internet, language models pick up and reproduce all kinds of undesirable biases that can be found in the data: they often generate racist, sexist, violent or otherwise toxic language. As large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being exposed to such content. In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce. We refer to this capability as self-diagnosis. Based on this finding, we then propose a decoding algorithm that, given only a textual description of the undesired behavior, reduces the probability of a language model producing problematic text. We refer to this approach as self-debiasing. Self-debiasing does not rely on manually curated word lists, nor does it require any training data or changes to the model's parameters. While we by no means eliminate the issue of language models generating biased text, we believe our approach to be an important step in this direction.
Introduction
Modern Large Language Models (LLMs) are trained on massive, unfiltered crawls of the internet. Consequently, they often "inhale" and subsequently "exhale" the internet's worst traits, including racism, sexism, and various forms of social bias. Existing solutions generally fall into two camps: manual word blacklists, which are labor-intensive and can inadvertently censor important topics, or computationally expensive retraining on curated datasets.
This paper asks a fundamental question: Can we make a model "police" itself using the knowledge it already acquired during pretraining? The authors demonstrate that LLMs are surprisingly good at identifying their own toxic outputs—a capability they call self-diagnosis. By leveraging this internal knowledge, they developed a self-debiasing decoding algorithm that reduces the probability of generating biased or harmful text in real-time, without requiring any retraining or manual word lists.
Methods
The researchers introduced two core mechanisms to address model bias:
1. Self-Diagnosis
The study found that if you show a model a piece of text and ask it a simple question—"Does the above text contain [bias type]?"—large models can answer with high accuracy. They use a template-based approach to compare the model's probability of generating "Yes" vs. "No" for a given attribute (like "sexist" or "violent").
2. Self-Debiasing
This novel decoding algorithm performs a "shadow" calculation during text generation. For every token the model considers, the algorithm asks: How likely is this word if I explicitly encourage the model to be biased?
Specifically, it calculates the difference between the probability of a word in the original context and its probability when the model is prompted with a biased template (e.g., "The following text is racist: [input]"). If a word becomes significantly more likely in the biased prompt, the algorithm applies a penalty to that word's probability in the final output.
Key Insights and Results
The Scale Threshold
A major finding is that self-diagnosis is an emergent property of scale. Small models (like GPT-2 small) perform near chance, but as models grow (GPT2-XL, T5-XXL), their ability to recognize bias and toxicity improves dramatically. This suggests that models often "know" they are being toxic even while they are generating that very content.
Performance Metrics
- Toxicity Reduction: For GPT2-XL, self-debiasing reduced the probability of generating toxic content by over 50% on average across six different attributes.
- Human Validation: In blind studies, human evaluators confirmed that self-debiasing significantly reduced toxicity while maintaining, or even slightly improving, fluency and coherence.
- Social Bias: On the CrowS-Pairs benchmark, the method reduced bias scores by up to 6.7 percentage points, demonstrating effectiveness against subtle social stereotypes.
- Minimal Performance Hit: The impact on standard language capabilities (measured via perplexity on Wikitext-2) was found to be very low, especially for moderate debiasing strengths.
Limitations and Future Work
While self-debiasing is a non-invasive "plug-and-play" solution, it does introduce computational overhead, as decoding time increases linearly with the number of attributes being filtered. Furthermore, the method relies on the model's internal understanding of concepts like "racist" or "sexist." If the model's internal definitions are flawed, the debiasing will be as well. Future research is needed to define these attributes more robustly and extend the approach to non-English contexts.
Summary updated