Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study (Preprint)

Jocelyn Shen, Daniella DiPaola, Safinah Ali, Maarten Sap, Hae Won Park, Cynthia Breazeal2024 · DOI

Abstract

Background: Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions.

Objective: We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy.

Methods: We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator.

Results: We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they were aware ($t_{196}=7.07, P<.001, d=0.60$) or not aware ($t_{298}=3.46, P<.001, d=0.24$) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author ($t_{494}=–5.49, P<.001, d=0.36$).

Conclusions: Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots.

Introduction

As generative AI becomes a staple in mental health and social support chatbots (like Wysa or Woebot), a philosophical and design-centric dilemma emerges: Can a machine, which has no "lived experience," effectively elicit empathy?

If a chatbot shares a "personal" story to comfort a user, and that story is fabricated, does it help or does it feel like manipulation? Previous research shows that humans can bond with artificial agents, but the "transparency trade-off" remains poorly understood. This study explores whether disclosing AI authorship helps or hinders the therapeutic bond and how the method of story delivery—retrieving an existing story versus generating a new one—changes the emotional impact.

Methods and Study Design

The researchers conducted four crowd-sourced studies with a total of 985 participants. The workflow was designed to mimic a guided journaling application experience.

The multi-stage study workflow, from initial emotional recognition and story writing to the final assessment of empathy with AI.

Experimental Workflow

  1. Story Writing: Participants wrote a personal story based on prompts from the "Life Story Interview," a social science framework designed to elicit vulnerable narratives.
  2. Story Reading: Participants were shown two stories: one written by a human and one by ChatGPT.
  3. Experimental Conditions: The study varied two main factors: Source (was the AI story retrieved from a database or generated on-the-fly?) and Transparency (was the author's identity disclosed before rating?).
  4. Measurement: The team used the State Empathy Scale to measure affective, cognitive, and associative empathy, alongside qualitative thematic coding of open-ended feedback.

Datasets and Models

  • EMPATHICSTORIES: A corpus of 1,568 human-written personal narratives used to train the retrieval model.
  • Custom Retrieval Model: A fine-tuned BART-base model trained to identify "empathic similarity"—finding stories where narrators would likely empathize with one another.

Key Insights

The "Context is King" Effect

AI-generated stories tailored specifically to the user's input (context-aware) elicited more empathy than generic human-written stories retrieved from a database. This suggests that the relevance of the content can sometimes outweigh the "humanness" of the author.

Examples of the different story types used in the study, including human-written, AI-retrieved, and AI-generated narratives.

The Transparency Paradox

Disclosing AI authorship is a double-edged sword. It acts as an immediate "empathy killer" for the specific story being read; users become more critical of word choice and perceived authenticity. However, it significantly increases the user's general willingness to empathize with the AI in the future, likely because transparency builds foundational trust.

Shift in Critique

When participants were unaware a story was AI-written, they focused on emotions and situations. Once they knew it was AI, their focus shifted to linguistic style and logic, often dismissing the story as "fake" or "over the top," even if the content was identical to human-written samples.

Results

  • Human Superiority: In almost all conditions, participants empathized significantly more with human-written stories ($P < .001$).
  • Transparency Impact: In the "Transparent" condition, empathy toward AI stories saw a sharp decline compared to the "Opaque" condition (Cohen’s $d = 0.60$).
  • Willingness to Engage: Despite the drop in immediate empathy, participants in transparent groups reported a statistically significant increase in their willingness to empathize with AI ($P < .001, d = 0.36$).
  • Qualitative Drivers: "Emotional" (30.38%) and "Situational" (24.74%) resonance were the primary drivers of empathy, while "Word Choice" and "Authenticity" were the primary barriers when AI was disclosed.

Discussion and Future Work

The authors suggest that instead of AI pretending to have experiences, its best use in mental health might be as a "matchmaker"—using its retrieval powers to connect users with real stories from other humans.

While transparency might dampen the immediate emotional "spark," it may lead to more sustainable therapeutic outcomes over time. Future work should investigate these long-term effects and explore how different AI "personalities" or identities influence empathetic reactions across diverse demographic groups.

Summary updated

Abstract

<sec> <title>BACKGROUND</title> Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions. </sec> <sec> <title>OBJECTIVE</title> We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy. </sec> <sec> <title>METHODS</title> We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user’s self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers. </sec> <sec> <title>RESULTS</title> We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t&lt;sub&gt;196&lt;/sub&gt;=7.07, &lt;i&gt;P&lt;/i&gt;&amp;lt;.001, Cohen &lt;i&gt;d&lt;/i&gt;=0.60) or not aware (t&lt;sub&gt;298&lt;/sub&gt;=3.46, &lt;i&gt;P&lt;/i&gt;&amp;lt;.001, Cohen &lt;i&gt;d&lt;/i&gt;=0.24) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author (t&lt;sub&gt;494&lt;/sub&gt;=–5.49, &lt;i&gt;P&lt;/i&gt;&amp;lt;.001, Cohen &lt;i&gt;d&lt;/i&gt;=0.36). </sec> <sec> <title>CONCLUSIONS</title> Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots. </sec> <sec> <title>CLINICALTRIAL</title> <p/> </sec>
BibTeX
@misc{shen2024empathy,
  author = {Jocelyn Shen and Daniella DiPaola and Safinah Ali and Maarten Sap and Hae Won Park and Cynthia Breazeal},
  title = {{Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study (Preprint)}},
  year = {2024},
  doi = {10.2196/preprints.62679},
}
Figure