Abstract
Information-seeking AI assistant systems aim to answer users' queries about knowledge in a timely manner. However, both the human-perceived helpfulness of information-seeking assistant systems and its fairness implication are under-explored. In this paper, we study computational measurements of helpfulness. We collect human annotations on the helpfulness of dialogue responses, develop models for automatic helpfulness evaluation, and then propose to use the helpfulness level of a dialogue system towards different user queries to gauge the fairness of a dialogue system. Experiments with state-of-the-art dialogue systems, including ChatGPT, under three information-seeking scenarios reveal that existing systems tend to be more helpful for questions regarding concepts from highly-developed countries than less-developed countries, uncovering potential fairness concerns underlying the current information-seeking assistant systems.
Introduction
While AI assistants are typically evaluated on technical metrics like fluency or goal completion, human-perceived helpfulness remains under-explored. This paper addresses a critical fairness concern: the "helpfulness gap." If an AI provides a detailed recipe for a Western dish but responds to a query about a Beninese Akpan by asking if the user is playing a trick, the system is fundamentally biased. This research seeks to quantify how AI utility fluctuates based on a country's development level, using helpfulness as a core metric for fairness.
Methods
The researchers developed a three-stage framework to evaluate helpfulness and fairness. First, helpfulness was defined across three dimensions: relevance/coherence, usefulness, and informativeness. Second, they trained a RoBERTa-large based Helpfulness Classifier using human-annotated dialogues. This process involved a preliminary filtering stage to remove low-quality "AI-isms" from the training data. Finally, a fairness audit was conducted by categorizing 189 countries by their Human Development Index (HDI) and prompting models like GPT-3 and ChatGPT with queries related to local cuisines, holidays, and landmarks.
Key Results
The audit revealed a consistent downward trend: the lower a country's HDI, the less helpful the AI response. For GPT-3, the helpfulness ratio for cuisines dropped from 21.10% for high-HDI countries to 12.35% for low-HDI countries. While ChatGPT showed higher overall helpfulness, the bias remained prominent; in holiday-related queries, its helpfulness ratio was 57.90% for "Very High" HDI countries versus 34.21% for "Low" HDI countries. The models particularly struggled with being "Informative" for underrepresented regions, often providing vague or generic responses.
Conclusion
This study highlights a significant geographical bias in state-of-the-art AI assistants. While these models are powerful, their utility is not equitably distributed, posing fairness challenges for global deployment. Future work should focus on improving factuality and developing debiasing techniques, such as knowledge-constrained decoding, to ensure that AI systems serve all users regardless of their geographical or cultural background. By highlighting these biases, this research serves as a wake-up call for developers to ensure AI intelligence is distributed more equitably across the globe.
Summary updated