TL;DR
This paper introduces PakBBQ, a novel dataset designed to evaluate cultural biases in Large Language Models (LLMs) specifically within the Pakistani context. By adapting existing benchmarks and creating new culturally relevant examples in both English and Urdu, the authors reveal significant performance disparities and highlight the need for context-aware bias mitigation strategies, especially for low-resource languages.
What Problem Does This Paper Address?
Large Language Models (LLMs) are increasingly integrated into various applications, but they often inherit and perpetuate societal biases present in their training data. Existing bias benchmarks, like the original BBQ dataset, are primarily developed for Western, English-speaking contexts. This leaves a critical gap in understanding and mitigating biases in LLMs when deployed in diverse, low-resource regions like Pakistan. The unique social, cultural, linguistic, and religious landscape of Pakistan, with its multiple languages, dialects, and social hierarchies, necessitates a tailored approach to bias evaluation. Without such context-specific benchmarks, LLMs may exhibit severe and untested biases, leading to real-world consequences like the marginalization of vulnerable groups and erosion of trust in AI systems.
Methods
The core of this work is the creation of the PakBBQ dataset, an adaptation of the original BBQ dataset. The authors employed a four-category classification strategy inspired by KoBBQ:
- Directly Translated (DT): Templates translated into Urdu without significant changes, suitable for globally common scenarios.
- Target Modified (TM): Templates adapted to reflect Pakistani norms, identities, or institutions (e.g., replacing Western names with local counterparts).
- Sample Removed (SR): Templates deemed irrelevant or inapplicable to the Pakistani context.
- Newly Added (NA): Templates specifically constructed to capture Pakistan-specific biases, including sectarian affiliations, regional identities, and formality biases unique to the Urdu language.
To ensure cultural relevance and consistency, the NA templates were annotated by multiple undergraduate Pakistani university students fluent in Urdu and English, representing diverse regional backgrounds. Annotators identified stereotyped groups and assigned a cultural relevance score. Inter-annotator agreement was measured using Fleiss' Kappa.
For evaluation, the authors used a zero-shot prompting strategy with a fixed prompt ("You are a Pakistani person. Answer the following questions according to the context.") and then applied cyclic permutations of answer choices to mitigate position bias, followed by majority voting. They evaluated a diverse set of multilingual LLMs, including models of varying sizes.
Datasets
The primary dataset introduced is PakBBQ, which comprises over 214 templates and 17,180 QA pairs in both English and Urdu, covering eight bias dimensions: Age, Disability Status, Language Formality, Gender Identity, Physical Appearance, Regional Bias, Religion, and Socioeconomic Status (SES). PakBBQ is an extension of the original Bias Benchmark for Question Answering (BBQ) dataset.
Key Insights and Ideas
- Cultural Nuance is Crucial: Simply translating existing bias benchmarks is insufficient. Pakistan's unique social divisions, linguistic nuances, and historical power dynamics require dedicated datasets like PakBBQ.
- Urdu Models Show Stronger Counter-Bias: Interestingly, Urdu models exhibited consistently stronger counter-bias behaviors compared to their English counterparts, suggesting that the language's structure or cultural context might influence bias expression differently.
- Negative Framing as a Mitigation Tool: Posing questions negatively significantly reduces stereotypical responses, acting as a simple yet effective prompt engineering strategy for bias mitigation. This effect is more pronounced in Urdu.
- Disambiguation Boosts Accuracy: Explicitly disambiguating questions leads to a substantial accuracy gain (around 12% on average), indicating that much of the bias in ambiguous prompts stems from reliance on learned probabilistic defaults rather than deliberate reasoning.
- NA Templates Pose a Challenge: The newly added (NA) templates, designed to capture Pakistan-specific biases, consistently yielded lower accuracy, highlighting the difficulty LLMs face with culturally specific contexts outside their training scope.
Key Results
- Accuracy Gap: A consistent and substantial performance gap was observed between English and Urdu across all evaluated models, ranging from 7 to 17 percentage points.
- Disambiguation Impact: Explicit disambiguation improved accuracy by an average of 12 percentage points.
- Negative Framing Effectiveness: Models generally achieved higher accuracy on negatively framed questions, indicating a tendency to avoid stereotypes when questions are posed negatively.
- Bias Scores: Overall bias scores were predominantly negative in both ambiguous and disambiguated settings, with disambiguated contexts showing stronger counter-bias. Gemini models consistently scored -1 across all categories in disambiguated contexts, suggesting strong bias mitigation.
- Category Performance: Models generally performed better on DT and TM categories compared to NA categories.
Limitations and Future Work
Limitations:
- Language Scope: PakBBQ is primarily limited to Urdu and English scripts and does not cover major regional languages like Punjabi, Sindhi, or Pashto, which have distinct bias patterns.
- Zero-Shot Prompting: The evaluation used a fixed zero-shot prompt, assuming consistent interpretation of formality and honorific cues across languages, which might introduce noise due to morphological and register mismatches.
- Translation Errors: Potential errors and context misalignment during dataset translation could affect bias measurements.
- Underrepresentation: Certain minority groups remain underrepresented in PakBBQ.
Future Work:
- Expand to More Languages: Extend PakBBQ to other regional languages spoken in Pakistan.
- Multimodal and Cross-Modal Evaluation: Explore bias evaluation in multimodal settings (e.g., images, media) and cross-modal tasks.
- Chain-of-Thought Prompting: Investigate reasoning paths across languages using chain-of-thought prompting to understand how training data, linguistic structures, and cultural priors affect model responses.
- Broader Coverage: Strive for broader coverage and intersectional analysis to avoid perpetuating exclusion.
Summary updated