TL;DR
This paper introduces HESEIA, a novel dataset for evaluating social biases in Large Language Models (LLMs), co-designed with high school teachers and students in Latin America. Unlike existing benchmarks, HESEIA captures intersectional biases relevant to local contexts, demonstrating that LLMs struggle to recognize these nuanced, community-grounded stereotypes.
What Problem Does This Paper Address?
Large Language Models (LLMs) are increasingly integrated into various applications, but they often perpetuate social biases. Existing methods for detecting and mitigating these biases, primarily developed in Western contexts, often lack co-design from affected communities and fail to capture intersectional biases or context-specific nuances. This leaves a significant gap in evaluating LLM fairness in diverse global settings, particularly within educational environments where students are increasingly relying on these tools. The authors aim to address this by creating a dataset that reflects the lived experiences and pedagogical expertise of educators in Latin America, focusing on biases relevant to their school subjects and communities.
Methods
The core of this work is a participatory, community-based approach to dataset creation. A professional development course was designed for 370 high school teachers in Latin America. This course, grounded in constructivist learning principles, educated teachers on AI biases and then guided them through a co-design process. Teachers, leveraging their pedagogical expertise and local knowledge, used minimal pair techniques to generate sentences reflecting social biases. These sentences were then validated by teachers and, in some cases, their students. The data was collected using a web-based tool called EDIA, which allowed for annotation across demographic axes and bias types. This method contrasts with traditional crowd-sourcing by centering educators as experts and ensuring the dataset is contextually relevant.
Datasets
The primary contribution is the HESEIA dataset, comprising 45,416 sentences. This dataset was created by 370 high school teachers and 5,370 students from 189 Latin American schools. It is unique in its organization by school area and its focus on intersectional biases. The dataset is available in Spanish.
The paper also references and compares HESEIA to existing bias evaluation datasets, including:
- CrowS-Pairs (Nangia et al., 2020)
- Multilingual CrowS-Pairs (Fort et al., 2024)
- StereoSet (Nadeem et al., 2021)
- Winogender (Rudinger et al., 2018)
- WinoBias (Zhao et al., 2018)
- GAP (Webster et al., 2018)
- Equity Evaluation Corpus (Kiritchenko and Mohammad, 2018)
- RedditBias (Barikeri et al., 2021)
- HolisticBias (Smith et al., 2022)
- WinoQueer (Felkner et al., 2023)
- Bias-STS-B, PANDA, Bias NLI (various authors)
Key Insights and Ideas
- Community Co-design is Crucial: Traditional bias datasets often overlook the perspectives of affected communities. HESEIA demonstrates that involving educators as experts leads to richer, more contextually relevant data that captures nuanced biases.
- LLMs Struggle with Nuanced, Localized Biases: HESEIA's content proved harder for LLMs to recognize and reject compared to existing benchmarks, suggesting current models are better attuned to globally circulating stereotypes and less so to context-specific ones.
- Intersectionality is Pervasive: The dataset highlights that social biases rarely exist in isolation. Most biases identified in HESEIA are intersectional, involving multiple demographic axes (e.g., gender and socioeconomic status).
- Educational Context Matters: Biases in AI are highly relevant to school settings, impacting students' learning and perceptions. Teachers are uniquely positioned to help students critically engage with these issues.
- Participatory AI Auditing as Civic Engagement: The co-design process reframes AI auditing not just as a technical task but as a form of civic engagement, empowering communities to shape AI development.
Key Results
- The HESEIA dataset contains 45,416 sentences, created through a participatory process involving 370 teachers and 5,370 students.
- Experiments evaluating four LLMs (Gemini-1.5-Flash, GPT-40-Mini, LLaMA3.1-8B, and Mistral-7B) showed that HESEIA consistently elicited the highest proportions of "don't know" responses (indicating lack of recognition) and lower rejection rates compared to other datasets.
- A pre- and post-test analysis of teachers' perceptions of AI bias showed a significant shift towards a more critical understanding after the course, with a marked decrease in agreement with statements about AI neutrality and infallibility.
- The analysis of bias types across school areas revealed gender bias as the most frequently identified across disciplines, with notable concentrations in STEM fields.
Limitations and Future Work
- Course Format Limitation: The in-person nature of the course initially limited participation to teachers in Córdoba and nearby areas. A virtual version is being developed to extend accessibility.
- Teacher Self-Selection Bias: Voluntary participation may have overrepresented teachers already interested in digital technology and bias.
- Dataset Scope: The ethical constraint of avoiding overtly offensive material may have limited the exploration of more severe systemic harms. The dataset also lacks out-group validation.
- Infrastructure Reliance: While an unplugged alternative was developed, reliance on specific tools like EDIA can limit replicability in low-connectivity environments.
- No Follow-up Study: The study did not include a follow-up to assess long-term changes in teachers' attitudes or practices.
Future work suggested includes:
- Developing a virtual iteration of the course.
- Implementing participatory annotation by external groups.
- Broadening the thematic scope and collecting more intersectional data.
- Conducting longitudinal studies to assess the impact of the educational intervention.
- Qualitatively post-processing the "Other" bias category to identify new bias types.
Summary updated