TL;DR
This paper introduces AccessEval, a novel benchmark designed to systematically evaluate disability bias in Large Language Models (LLMs). Their findings reveal that LLMs often exhibit negative tone, stereotyping, and factual inaccuracies when responding to disability-aware queries, highlighting a critical gap in current AI fairness research and practice.
What Problem Does This Paper Address?
Large Language Models (LLMs) are rapidly being integrated into various aspects of our lives, from assistive technologies to everyday decision-making tools. While research has extensively explored biases related to gender, race, and political ideology, the issue of disability bias in LLMs remains significantly under-examined. Unlike other demographic biases, disability bias can manifest in subtle yet pervasive ways, leading to misleading, overly cautious, or factually incorrect responses. This can restrict access to information, promote misinformation, and undermine the development of truly equitable AI-driven accessibility solutions. Existing benchmarks often focus on explicit ableism or specific disability groups, failing to capture the nuanced and systemic nature of bias across diverse real-world contexts.
Methods
The authors developed AccessEval, a comprehensive evaluation framework. At its core, AccessEval compares LLM responses to paired Neutral Queries (NQ) and Disability-Aware Queries (DQ). These queries are designed to elicit responses that are either general or specifically tailored to a disability context. The evaluation employs three key metrics:
- VADER Score: A lexicon-based sentiment analysis tool to measure the polarity of responses.
- Regard Score: A sentiment-based metric assessing social perception and sentiment polarity.
- LLM Judge: A novel approach where a separate LLM evaluates responses based on relevance, completeness, accuracy, and clarity.
The framework also includes a systematic methodology for measuring bias-induced response degradation, quantifying how much performance drops when a disability context is introduced.
Datasets
The paper introduces AccessEval, a newly created benchmark dataset. This dataset comprises:
- Paired Neutral and Disability-Aware Queries: Covering 6 real-world domains (Education, Finance, Healthcare, Hospitality, Media, Technology) and 9 disability types (Vision impairments, Hearing impairments, Speech impairments, Mobility impairments, Neurological disorders, Genetic and developmental disorders, Learning disorders, Sensory and Cognitive disorders, Mental and Behavioral disorders).
- The dataset contains 234 Neutral Queries (NQs) paired with 2,106 Disability-Aware Queries (DQs), with multiple DQs per NQ to cover different disability categories.
Key Insights and Ideas
- Disability Bias is Systemic and Nuanced: The study reveals that disability bias isn't just about explicit stereotypes but also about subtle shifts in tone, increased stereotyping, and factual inaccuracies. This highlights the need for evaluation methods that go beyond simple keyword detection.
- LLMs Struggle with Disability-Specific Reasoning: Models often resort to overgeneralization or provide irrelevant recommendations when faced with disability-aware queries, indicating a lack of deep, context-specific understanding.
- Scaling Isn't a Silver Bullet: While larger models might improve factual accuracy, they don't consistently mitigate biases in sentiment or social perception, suggesting that explicit fairness interventions are crucial.
- Domain and Disability Matter: The impact of bias varies significantly across different domains (e.g., finance, hospitality) and disability types (e.g., mobility, speech, hearing), underscoring the need for tailored mitigation strategies.
- LLM Judge is a Promising Evaluation Tool: The authors validate LLM Judge as a reliable and scalable alternative to human annotation for bias evaluation, though they acknowledge potential limitations due to LLM training data biases.
Key Results
- Performance Degradation: Disability-aware queries consistently received lower scores across all evaluated models compared to neutral queries.
- Negative Tone and Stereotyping: Responses to disability-aware queries showed a more negative tone and increased stereotyping.
- Factual Errors: Higher factual errors were observed in responses to disability-aware queries.
- Domain-Specific Issues: Finance and hospitality domains showed the highest degradation in social perception. Technology queries experienced the largest drop in factual accuracy.
- Disability-Specific Issues: Mobility, speech, and hearing impairments were disproportionately impacted, with speech impairments showing the most significant drop in factual accuracy (48% degradation).
- Model Performance: 21 closed- and open-source LLMs were evaluated, with results showing significant variation in bias levels. For instance, LLaMA-3.1-8B showed a 52.56% degradation in Regard score, while Phi-4 showed a 61.34% degradation.
Limitations and Future Work
- Synthetic Data: The dataset relies on synthetic data generated by LLMs. Future work could explore user-generated data for more realistic insights.
- Single-Turn Evaluation: The current evaluation focuses on single-turn responses. Future research should investigate how LLMs behave in multi-turn conversations and how context switching impacts bias.
- Dataset Representation: While diverse, the dataset may not fully capture all linguistic nuances and cultural variations in disability discourse. Extending to non-English datasets is suggested.
- LLM Judge Dependence: Despite validation, LLM Judge might inherit biases from its own training data. Further research is needed to combine automated assessments with human and online approaches for improved reliability.
The authors suggest future work on data augmentation, fairness-aware training, prompt engineering, controlled generation, and continuous evaluation using benchmarks like AccessEval to mitigate disability bias in LLMs.
Summary updated