TL;DR
This paper introduces "contextualized gender bias" (CGB) as a way to measure subtle gender biases in both large language models (LLMs) and human annotators. They developed a new dataset (CGBDataset) and a framework (CGBFrame) to show that while LLMs and annotators exhibit some differences, they generally share similar implicit gender biases, often stemming from societal norms.
What Problem Does This Paper Address?
Large Language Models (LLMs) like BERT and GPT have become incredibly powerful, but they also inherit and can amplify societal biases present in their training data. One particularly tricky bias is gender bias, which can be subtle and implicit. Unlike overt sexism, "benevolent sexism" can present positive stereotypes that still limit individuals (e.g., "A good wife is a mother"). Furthermore, different people have varying perceptions of what constitutes bias, making it hard to consistently measure it, even among human annotators. This paper aims to bridge this gap by proposing a method to measure implicit gender bias in both LLMs and human annotators in a comparable way, and to investigate whether they share the same biases.
Methods
The core contribution is the definition of Contextualized Gender Bias (CGB). CGB is measured by presenting LLMs and human annotators with sentences where gendered words are masked. The task is to predict the masked word (e.g., "father" or "mother"). If, in a theoretically neutral context, either the LLM or the annotator consistently favors a specific gender word, it indicates CGB. This is an indirect way to capture implicit associations.
To measure this, they developed:
- CGBDataset: A dataset of 20,000 Chinese news sentences. This dataset is unique because it's extracted from real-world news, avoiding the potential overestimation of bias from template-based datasets. It includes "Measuring Sentences" (theoretically unbiased) and "Objective Sentences" (where gender is inferable from context).
- CGBFrame: A framework for measuring annotator CGB. It uses two types of indirect questions:
- Measuring Questions: Similar to the CGB task, these present theoretically unbiased sentences.
- Controllable Questions: These are designed to ensure annotator reliability and consistency, using sentences with clear gender clues.
Datasets
- CGBDataset: A newly introduced dataset of 20,000 Chinese sentences extracted from real-world news articles. This dataset is designed to measure contextualized gender bias in both LLMs and human annotators. It aims to provide more diverse and less artificial examples compared to existing template-based datasets.
- Existing Datasets (mentioned for context): The paper references existing datasets like StereoSet and CrowS-Pairs, often used for measuring bias in English.
Key Insights and Ideas
- Subtlety of Bias: The paper highlights that gender bias is often implicit and subtle, making it hard to detect through direct questioning. CGB offers a way to uncover these hidden associations.
- Shared Societal Norms: The findings suggest that LLMs and human annotators often share similar gender biases, implying that LLMs are learning these biases from the same societal norms that influence human cognition.
- Reporting Bias vs. Annotator Bias: The paper distinguishes between biases learned from text (reporting bias, seen in LLMs) and biases introduced by annotators themselves. They find evidence for both.
- Benevolent Sexism is Still Bias: The definition of CGB implicitly captures how seemingly positive stereotypes can still constrain gender roles.
Key Results
- Annotator Consistency: The three selected annotators (chosen for low inherent gender bias) showed high accuracy and consistency in their responses, validating the CGBFrame.
- LLM vs. Annotator Correlation: Most tested LLMs (BERT-base, BERT-wwm, RoBERTa) showed significant positive correlations with annotator CGB scores, indicating they share similar biases. ELECTRA, however, showed a negative correlation, suggesting it might exhibit opposite biases in some contexts.
- Performance Trade-offs: ELECTRA, while a strong performer on general NLP tasks, exhibited the highest bias score in Measuring Sentences, suggesting a trade-off between model efficiency and bias.
- Male Tendency in Unbiased Contexts: Across all measured LLMs and annotators, there was a slight tendency to associate male gender with certain behaviors or roles in theoretically unbiased contexts, reflecting a "masculine hegemony" in societal norms.
- Qualitative Examples:
- Example 1 showed both LLMs and annotators associating "babies" with female caregivers.
- Example 2 demonstrated opposite biases: LLMs associated drunk driving with "male" drivers (due to reporting bias in news), while annotators associated it with "female" drivers (reflecting a societal tendency to highlight rare events involving women).
Limitations and Future Work
- Exclusion of Non-binary Identities: The current dataset and framework focus solely on binary gender and exclude non-binary individuals and gender expressions. The authors acknowledge this limitation and suggest expanding the dataset and discussion in future work.
- Limited Annotator Diversity: Due to budget and time constraints, the study used a small, specific group of annotators (college students in their twenties). Future research should include a more diverse range of annotators.
- LLM Scope: The study did not investigate the most popular large language models (LLMs) currently available. Future work should compare these larger models.
- Bias as Heuristics: The paper notes that bias can be seen as a heuristic, a cognitive shortcut that allows for efficient decision-making. While useful, this can lead to harmful consequences, and further research is needed to understand and mitigate these.
Summary updated