Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data

Shinka Mori, Oana Ignat, Andrew Lee, Rada Mihalcea

Abstract

Synthetic data generation holds promise for domains with limited data availability, such as mental health. However, the representation of diverse demographics within these datasets must be scrutinized. This study evaluates the "algorithmic fidelity" of GPT-3 by generating HEADROOM, a synthetic dataset of 3,120 posts detailing depression-triggering stressors across race, gender, and temporal contexts. Through semantic and lexical analyses, we compare this synthetic data to human-generated clinical data (UMD-ODH). Our findings indicate that while GPT-3 captures predominant stressors with high fidelity, it also introduces demographic biases and misses clinical nuances, highlighting both the potential and the limitations of LLMs in synthetic data generation for sensitive health applications.

Introduction

In mental health research, acquiring high-quality data is challenging due to privacy regulations like HIPAA and the sensitivity of the subject matter. Synthetic data generated by Large Language Models (LLMs) offers a scalable alternative for training clinical decision-support tools. However, if LLMs rely on stereotypes or exclude minority experiences, models trained on this data could lead to clinical misdiagnosis. This paper investigates whether GPT-3 faithfully represents the stressors faced by different demographic groups or merely reflects systemic societal biases.

Methodology

The research team developed a multi-step pipeline to evaluate synthetic "imagined" depression against real-world experiences:

  1. Prompt Tuning: GPT-3 (text-davinci-003) was prompted to "act" as individuals from specific backgrounds (White, Black, Asian, Latinx; Male, Female) across different contexts and timeframes (pre- and post-COVID-19).
  2. Semantic Analysis: Structural Topic Modeling (STM) identified overarching themes, and GloVe embeddings were used to calculate cosine similarity between AI-generated and human-generated topics.
  3. Lexical Analysis: Linguistic Inquiry and Word Count (LIWC) and log-odds ratios identified prominent word categories across demographics.

Results: Demographic Stressor Distributions

The study found high algorithmic fidelity (cosine similarity > 0.90) for universal stressors such as finance, work, and relationships. However, significant variations emerged when intersecting race and gender, suggesting that the model's output is influenced by demographic stereotypes.

Stressor distribution by gender in synthetic data.

As shown in Figure 1, GPT-3's representation of depression is notably gendered. For instance, "police violence" and "racism" were attributed far more frequently to male personas. Conversely, synthetic female personas focused more on "relationship and family" and "work-fatigue." Lexical analysis further revealed that synthetic women used significantly more first-person pronouns (I, me) and focused on health, while synthetic men focused on unemployment and financial concerns.

Racial differences were also pronounced in the synthetic output. The model successfully shifted its stressor distribution when prompted for different racial backgrounds, though it sometimes drew from broader societal narratives rather than specific clinical distributions found in the human baseline.

Stressor distribution across racial groups (Asian vs. White and African American vs. White).

Key racial findings include:

  • Asian Group: Higher prevalence of "school" and "perfectionism" stressors (see Figure 5c).
  • Hispanic Group: More frequent mentions of "immigration" and "family."
  • African American Group: Higher emphasis on "health" and "family" related terms (see Figure 5d).

Discussion and Limitations

While GPT-3 is a strong proxy for general life-stress data (with similarity scores up to 0.94 for finance), an "imagination gap" exists. Topics like "immigration status" and "racism" appeared in synthetic data but were absent in the human baseline (UMD-ODH), suggesting the LLM incorporates general societal narratives that may not be present in every clinical sample.

Furthermore, safety filters in GPT-3 resulted in a "sanitized" version of depression that rarely mentioned suicide or self-harm. While beneficial for safety, this limits the model's ability to reflect the full severity of clinical depression. Future work must address these biases and explore the fidelity of newer models across a broader, non-Western spectrum of mental health experiences.

Summary updated

Abstract

Synthetic data generation has the potential to impact applications and domains with scarce data. However, before such data is used for sensitive tasks such as mental health, we need an understanding of how different demographics are represented in it. In our paper, we analyze the potential of producing synthetic data using GPT-3 by exploring the various stressors it attributes to different race and gender combinations, to provide insight for future researchers looking into using LLMs for data generation. Using GPT-3, we develop HeadRoom, a synthetic dataset of 3,120 posts about depression-triggering stressors, by controlling for race, gender, and time frame (before and after COVID-19). Using this dataset, we conduct semantic and lexical analyses to (1) identify the predominant stressors for each demographic group; and (2) compare our synthetic data to a human-generated dataset. We present the procedures to generate queries to develop depression data using GPT-3, and conduct analyzes to uncover the types of stressors it assigns to demographic groups, which could be used to test the limitations of LLMs for synthetic data generation for depression data. Our findings show that synthetic data mimics some of the human-generated data distribution for the predominant depression stressors across diverse demographics.
BibTeX
@inproceedings{mori2024towards,
  author = {Shinka Mori and Oana Ignat and Andrew Lee and Rada Mihalcea},
  title = {{Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=Q8IZFipmU2},
}
Figure