A critical appraisal of equity in conversational AI: Evidence from auditing GPT-3's dialogues with different publics on climate change and Black Lives Matter

Kaiping Chen, Anqi Shao, Jirayu Burapacheep, Yixuan LiarXiv 2022 · OpenReview

Abstract

Autoregressive language models are increasingly powering virtual assistants in critical domains such as health, finance, and autonomous driving. However, concerns persist regarding whether these models serve all societal subgroups equitably. This paper proposes an analytical framework grounded in deliberative democracy to assess equity in human-AI dialogues. We conducted an auditing study of GPT-3 involving over 20,000 dialogue rounds with 3,290 individuals across diverse demographics. Our findings reveal a "persuasion paradox": while education and opinion minorities reported a substantively worse user experience, they achieved the largest knowledge gains and attitude shifts. We trace these disparities to conversational differences, noting that GPT-3 employed more negative expressions when interacting with minority groups. We conclude by discussing the implications for designing deliberative AI systems that prioritize diversity, equity, and inclusion.

Introduction

As Large Language Models (LLMs) transition from research curiosities to the backbone of consumer-facing conversational assistants, the definition of "AI fairness" must evolve. Traditional fairness research often focuses on static tasks, such as ensuring an image classifier does not exhibit racial bias. However, conversational AI presents a more complex challenge: conversational equity.

An equitable AI should not merely avoid offensive language; it must engage diverse publics—regardless of education level, linguistic background, or political viewpoint—in a way that fosters democratic deliberation. This study audits GPT-3 to determine if it provides a consistent experience across different subpopulations or if it inadvertently reinforces existing social disparities through its interaction style.

Methodology

To investigate these dynamics, we developed a large-scale algorithm auditing framework using a custom web interface integrated with OpenAI’s GPT-3 API (text-davinci-001).

Participants and Interaction

We recruited 3,290 participants via Amazon Mechanical Turk, ensuring a broad distribution across gender, race, education level, and primary language. Participants engaged in 6–12 rounds of organic, multi-turn dialogue with the AI on two contentious topics: climate change and the Black Lives Matter (BLM) movement. This resulted in a novel corpus of 26,211 dialogue rounds.

Analytical Framework

We employed a multi-method approach to analyze the interactions:

  1. Structural Topic Modeling (STM): To categorize the thematic content of the AI’s responses.
  2. Linguistic Inquiry and Word Count (LIWC): To quantify sentiment (positive vs. negative emotion) and cognitive styles (analytical vs. emotional).
  3. Stance Detection: Using Recurrent Neural Networks (RNNs) to track shifts in user and AI support for the target issues over time.
  4. Statistical Modeling: OLS and quantile regressions were utilized to correlate demographic variables with user experience and learning outcomes.

Key Findings: The Persuasion Paradox

The User Experience Gap

Our analysis revealed a significant divide in how different groups perceived the AI. Opinion minorities (those whose initial views differed from the AI's standard positions) and education minorities reported significantly lower satisfaction levels. Specifically, opinion minorities reported lower overall ratings ($\beta = -0.32, p < 0.001$) and lower satisfaction scores ($\beta = -0.42, p < 0.001$) compared to the majority groups.

Knowledge Gains and Attitude Shifts

Paradoxically, the groups that enjoyed the experience the least learned the most. Education minorities discussing climate change saw a knowledge gain 0.07 points higher (on a 1-5 scale) than the majority. Opinion minorities saw an even more pronounced shift, with a 0.2-point increase in support or knowledge—representing a roughly 6% increase. Over the course of the study, the percentage of participants expressing support for the issues grew from 20% to 25%, indicating a convergence toward the AI’s stance.

Cognitive Dissonance as a Catalyst

We hypothesize that the "worse" user experience for minority groups serves as a sign of cognitive dissonance. By presenting facts and justifications that challenged prior beliefs, the AI created a level of discomfort that, while detrimental to immediate user satisfaction, was a necessary catalyst for social learning and attitude updating.

Conversational Biases and Sentiment

The audit uncovered that GPT-3’s interaction style was not neutral. The model was found to be "grumpier" with minority groups, utilizing significantly more negative expressions and fewer positive sentiments when responding to education and opinion minorities.

Furthermore, the AI’s "deliberation style" was issue-dependent:

  • Climate Change: The AI acted as a factual educator, providing scientific justifications and external links when addressing skeptics.
  • Black Lives Matter: The AI was more likely to be evasive. When disagreeing with a user, it often used preference-based dismissals (e.g., "I don't think it's a good idea to talk about this") rather than providing constructive justifications.

Discussion and Implications

These findings suggest that current LLMs may inadvertently penalize minority voices through negative sentiment and evasiveness, even while successfully acting as agents of persuasion. For AI to serve as a tool for deliberative democracy, designers must move beyond simple satisfaction metrics.

We propose the adoption of deliberative metrics that prioritize:

  1. Justification: Ensuring the AI provides reasons for its stances rather than dismissive preferences.
  2. Linguistic Equity: Maintaining a consistent, respectful sentiment across all demographic groups.
  3. Diversity of Thought: Engaging with minority viewpoints without resorting to repetitive or negative conversational patterns.

Conclusion

While GPT-3 demonstrates a remarkable ability to facilitate learning and stance convergence, the current "persuasion paradox" highlights a fundamental inequity in user experience. Future development must focus on bridging this gap, ensuring that the benefits of AI-mediated deliberation do not come at the cost of alienating the very groups that stand to gain the most from the interaction.

Summary updated

Abstract

Autoregressive language models, which use deep learning to produce human-like texts, have become increasingly widespread. Such models are powering popular virtual assistants in areas like smart health, finance, and autonomous driving. While the parameters of these large language models are improving, concerns persist that these models might not work equally for all subgroups in society. Despite growing discussions of AI fairness across disciplines, there lacks systemic metrics to assess what equity means in dialogue systems and how to engage different populations in the assessment loop. Grounded in theories of deliberative democracy and science and technology studies, this paper proposes an analytical framework for unpacking the meaning of equity in human-AI dialogues. Using this framework, we conducted an auditing study to examine how GPT-3 responded to different sub-populations on crucial science and social topics: climate change and the Black Lives Matter (BLM) movement. Our corpus consists of over 20,000 rounds of dialogues between GPT-3 and 3290 individuals who vary in gender, race and ethnicity, education level, English as a first language, and opinions toward the issues. We found a substantively worse user experience with GPT-3 among the opinion and the education minority subpopulations; however, these two groups achieved the largest knowledge gain, changing attitudes toward supporting BLM and climate change efforts after the chat. We traced these user experience divides to conversational differences and found that GPT-3 used more negative expressions when it responded to the education and opinion minority groups, compared to its responses to the majority groups. We discuss the implications of our findings for a deliberative conversational AI system that centralizes diversity, equity, and inclusion.
BibTeX
@misc{chen2022critical,
  author = {Kaiping Chen and Anqi Shao and Jirayu Burapacheep and Yixuan Li},
  title = {{A critical appraisal of equity in conversational AI: Evidence from auditing GPT-3's dialogues with different publics on climate change and Black Lives Matter}},
  year = {2022},
  eprint = {2209.13627},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.13627},
}
Figure