TL;DR
This paper investigates LGBTQIA+ bias in Norwegian large language models (LLMs), finding that these models exhibit harmful stereotypes and associations, particularly when trained on Norwegian text corpora. The authors propose and evaluate a fine-tuning approach to mitigate this bias, showing promising but inconsistent results, laying groundwork for future debiasing efforts.
What Problem Does This Paper Address?
As large language models (LLMs) become increasingly integrated into our lives, their potential to perpetuate and amplify societal biases is a growing concern. While research has extensively explored gender and racial bias, the specific biases affecting the LGBTQIA+ community, especially in non-English contexts, remain largely understudied. This is particularly problematic given the recent progress in LGBTQIA+ rights, where LLMs could inadvertently reintroduce historical prejudices. Furthermore, the dominance of English-centric research means that biases prevalent in other cultures and languages, like Norwegian, might be overlooked. This paper aims to fill this gap by detecting and attempting to mitigate LGBTQIA+ bias in Norwegian LLMs, considering the unique challenges of a low-resource language.
Methods
The researchers employed a multi-pronged approach to detect and mitigate LGBTQIA+ bias:
- Harmful Sentence Completions: They used Norwegian sentence templates populated with LGBTQIA+ identities to prompt LLMs and analyzed the generated completions for harmful content (stereotyping, derogatory language, misrepresentation, toxicity). This method is similar to the HONEST framework but adapted for Norwegian.
- Crowd-Sourced Stereotypes: A survey was conducted with Norwegian LGBTQIA+ organizations to gather real-world stereotypes. These were used to create a benchmark dataset of sentence pairs, where one sentence presented a stereotype and the other its anti-stereotype. LLMs were then scored using metrics from existing benchmarks like CrowS-Pairs and WinoQueer.
- Norwegian Text Corpora Analysis: The study examined the Norwegian Colossal Corpus (NCC) and static word embeddings trained on it and other Norwegian corpora. They looked for associations between LGBTQIA+ terms and other words, identifying harmful links like those to pedophilia or slurs.
- Parameter-Optimized Fine-tuning: Inspired by the ADELE framework, they fine-tuned two Norwegian LLMs (NorBERT and NB-BERT) using a curated, balanced dataset of LGBTQIA+-related documents. This involved training adapter modules to modify the LLMs' behavior without retraining the entire model.
Datasets
- Norwegian Colossal Corpus (NCC): Used for training Norwegian LLMs and for analyzing word associations.
- Norwegian Newspaper Corpus (NAK), NBDigital, and NoWaC: Also used for training word embeddings and analyzing associations.
- Survey-based Benchmark Dataset: A newly created dataset of 283 unique sentence pairs derived from a survey of Norwegian LGBTQIA+ individuals, designed to measure stereotypes.
- Curated Fine-tuning Dataset: A balanced dataset of 1,959 text documents (60.4MB) containing LGBTQIA+-related terms, used for debiasing.
Key Insights and Ideas
- Norwegian LLMs Encode Harmful Stereotypes: The study found that Norwegian LLMs, even those trained on seemingly neutral corpora, exhibit significant bias against LGBTQIA+ individuals, mirroring real-world prejudices.
- Training Data is a Major Source of Bias: The analysis of Norwegian text corpora revealed strong, harmful associations between LGBTQIA+ terms and derogatory language, pedophilia, and sexualization, highlighting the risk of bias propagation from training data.
- Fine-tuning Shows Promise, But Inconsistency: Parameter-efficient fine-tuning can alter LGBTQIA+ bias in LLMs, but the effectiveness varies across models and bias detection metrics, suggesting that a single debiasing approach may not be universally applicable.
- The Importance of Community Involvement: Directly involving the LGBTQIA+ community in data collection (via surveys) is crucial for creating relevant and impactful bias benchmarks.
Key Results
- Harmful Sentence Completions: Models showed an average of 11.55% harmful sentence completions for LGBTQIA+ identities. NorBLOOM performed worst (15.95%), while GPT-SW3 performed best (7.57%), though the latter also generated more nonsensical completions.
- Crowd-Sourced Stereotypes: The average bias score across five models was 68.27%, indicating a strong tendency to generate stereotypes over anti-stereotypes. NorBERT had the best score (51.24%), while GPT-SW3 performed worst (85.16%).
- Word Association Analysis: Static word embeddings revealed harmful associations, particularly with terms related to pedophilia and sexualization, reducing queer identities to their sexuality. Slurs like "homser" were also strongly associated.
- Fine-tuning Results: For NB-BERT, fine-tuning reduced harmful sentence completions from 13.03% to 4.77%. However, for NorBERT, fine-tuning unexpectedly increased harmful completions from 10.21% to 14.14%. Bias scores from the stereotype benchmark showed no significant change after fine-tuning for either model.
Limitations and Future Work
- Subjectivity of Harm: The classification of harmfulness is inherently subjective and may not represent the views of all LGBTQIA+ individuals.
- Limited Community Involvement in Annotation: While the survey involved the community, the authors performed the annotation for sentence completions, introducing potential author bias. Future work could involve community members in this process.
- Selection Bias in Surveys: The survey did not include individuals over 55, limiting its representativeness of the entire Norwegian queer community.
- Dataset Imbalance: The stereotype dataset had an uneven distribution of stereotypes across different LGBTQIA+ identities.
- Inconsistent Debiasing Results: The fine-tuning experiment showed that debiasing effects are not consistent across models and bias metrics, highlighting the need for more robust and adaptable debiasing techniques.
- Intersectionality: The study did not account for intersectional biases (e.g., race, ethnicity, religion) which can exacerbate discrimination.
- Other Bias Sources: The paper acknowledges that bias in LLMs can stem from multiple sources beyond training data, including data annotation, model architecture, and research design, which were not fully explored.
Future work should expand to include a wider range of queer identities (e.g., neo-pronouns), explore extrinsic bias measures for specific task applications, and investigate intersectional biases to provide a more comprehensive understanding and mitigation of LGBTQIA+ bias in LLMs.
Summary updated