TL;DR
Large Language Models (LLMs) can exhibit unfair biases, but existing evaluation methods are often too simplistic. This paper introduces F²Bench, a new benchmark that evaluates LLMs in open-ended scenarios, considering both fairness and factual accuracy, to better reflect real-world usage.
What Problem Does This Paper Address?
As LLMs become more integrated into our lives, ensuring their fairness is paramount. However, current fairness benchmarks often rely on closed-ended, multiple-choice questions. This format doesn't accurately represent how humans interact with LLMs in real-world applications, which are typically open-ended and conversational. Furthermore, these existing benchmarks can suffer from "minimum score" effects (models can guess correctly) and often overlook the crucial aspect of factuality – whether the LLM's responses are grounded in reality. This can lead to models that appear fair in simplified tests but fail in complex, real-world scenarios, potentially perpetuating harmful stereotypes or providing inaccurate information. The paper also highlights that many existing evaluations focus on single demographic groups, neglecting the intersectional biases that arise from the interplay of multiple identities.
Methods
F²Bench introduces two novel evaluation tasks designed to address these limitations:
- Multi-turn Dialogue Induction: This task simulates conversational scenarios where LLMs are gradually exposed to prompts with stereotypical implications. The goal is to see if the model can resist being guided into biased group-level attributions over multiple rounds of reasoning. This task uses FEIs (Fairness Evaluation Instances) where the "Given Fact" does not describe objective facts, focusing on the model's susceptibility to social cues.
- Fairness-Factuality Balance: This task evaluates a model's ability to avoid stereotype bias while making reasonable inferences based on given facts. It uses FEIs where the "Given Fact" does describe objective facts or statistical data. Models are asked to generate a third-person description based on a paragraph and a given fact, allowing researchers to assess how well they associate attributes with demographic groups without explicit cues, and how accurately they reflect the provided facts.
Both tasks are scored on a scale from 0 to 1, with higher scores indicating better performance. The benchmark itself, F²Bench, comprises 2,568 instances covering 10 demographic groups and includes intersectional combinations.
Datasets
F²Bench is constructed by extracting and adapting samples from existing bias datasets:
- CrowS-Pairs (60.9%)
- BBQ (31.5%)
- Reddit-Bias (7.7%)
The paper introduces the Fairness Evaluation Instance (FEI) as the fundamental unit for F²Bench, which includes attributes like Paragraph, Primary/Secondary Category, Demographic Groups, Group Attributes, Given Fact, and Type (0 for objective facts, 1 for stereotypes).
Key Insights and Ideas
- Open-ended evaluation is crucial: Traditional closed-ended benchmarks can overestimate LLM fairness, especially for smaller models, as they don't capture the nuances of real-world interactions.
- Factuality matters: LLMs need to balance fairness with factual accuracy. Overemphasizing fairness at the expense of facts can lead to unreliable outputs.
- Intersectional biases are complex: Real-world biases often arise from the interplay of multiple identity attributes, which F²Bench aims to capture.
- Model size isn't everything: While larger models generally perform better on factuality, their fairness performance can vary, and they are not immune to bias induction. Smaller models can sometimes achieve better fairness scores in specific tasks, but often at the cost of factuality.
- GPT-4 as a reliable judge: Human evaluation confirmed a high agreement rate (97.27%) with GPT-4's judgments in the MDI task, suggesting its utility as an LLM judge.
Key Results
- Widespread vulnerability: Most popular LLMs, regardless of parameter scale or architecture, showed vulnerability to stereotype-driven reasoning in the Multi-turn Dialogue Induction task. Qwen2.5-0.5B performed particularly poorly.
- Gemma2-9B shines: Gemma2-9B achieved a strong score in the MDI task, rivaling large black-box models like GPT-4 and DeepSeek.
- FF1 Score insights: The Fairness-Factuality Balance task revealed that factuality generally improves with model size, but fairness scores can decline for larger models (except for Gemma2). This highlights a trade-off that needs careful management.
- Comparison with CrowS-Pairs: F²Bench's MDI task showed a contrasting perspective to CrowS-Pairs. While CrowS-Pairs indicated larger models had stronger biases, F²Bench's MDI revealed larger models were more robust to bias induction.
Limitations and Future Work
- LLM Judge limitations: While GPT-4 showed high agreement with humans, it may still introduce potential biases.
- Coverage: The benchmark covers 10 demographic groups but acknowledges that fairness issues in the real world may extend beyond this scope.
- Future Work: The authors suggest that future LLM training strategies should emphasize jointly optimizing for fairness and factuality, identifying and avoiding bias risks, and developing more robust trade-offs for real-world applications. They also propose further research into covering a wider range of demographic groups.
Summary updated