Disparities in seizure outcomes revealed by large language models

Kevin Xie, William K. S. Ojemann, Ryan S. Gallagher, Russell T. Shinohara, Alfredo Lucas, Chloe E. Hill, Roy H. Hamilton, Kevin B. Johnson, Dan Roth, Brian Litt, Colin A. Ellis

TL;DR

Researchers utilized a clinical Large Language Model (LLM) to analyze over 84,000 patient visits, revealing significant disparities in epilepsy outcomes: women and individuals with lower socioeconomic status are less likely to achieve seizure freedom, even when receiving specialty care. Crucially, the study first rigorously proved that the LLM itself was not intrinsically biased, ensuring the findings reflect real-world healthcare inequities rather than algorithmic error.

What Problem Does This Paper Address?

In the US healthcare system, it is well-documented that marginalized groups face barriers to accessing care. However, much less is known about disparities in outcomes—specifically, whether different groups achieve seizure freedom at different rates once they are already seeing a specialist.

The challenge in studying this at scale is that "seizure freedom" is usually buried in the unstructured text of clinical notes, making it nearly impossible for humans to audit tens of thousands of records. While Natural Language Processing (NLP) offers a solution, there is a significant risk that AI models might harbor "intrinsic bias," performing less accurately for certain demographic groups and thus producing skewed data. This paper tackles both the outcome gap and the AI trust gap.

Methods

The authors employed a multi-step pipeline to ensure both data scale and algorithmic fairness:

  • The Model: They used Clinical_BERT, a transformer-based language model pre-trained on clinical text, fine-tuned on 700 manually annotated epilepsy clinic notes.
  • Bias Auditing: Before analyzing the full dataset, they tested the model on a validation set of 192 notes (annotated in triplicate by humans). They calculated accuracy, Positive Class Balance (PCB), and Negative Class Balance (NCB) across demographics to ensure the model didn't "fail" more often for any specific group.
  • Statistical Analysis: They used logistic mixed-effects regression models. This approach allowed them to account for the fact that one patient might have many visits over 17 years (clustering) and to control for variables like the time between visits and the number of anti-seizure medications (ASMs) prescribed.

Datasets

  • Source: Electronic Health Records (EHR) from the University of Pennsylvania Health System spanning 2005–2022.
  • Scale: 84,675 clinic visits from 25,612 unique patients.
  • Demographics: Included self-reported race, ethnicity, and sex. Socioeconomic data was inferred by linking zip codes to US Census median household income and categorizing insurance types (Private vs. Public).

Key Insights and Ideas

  • The "Unbiased" AI: Perhaps the most technically reassuring finding is that the LLM performed with near-identical accuracy across all races, sexes, and income levels. This suggests that with proper validation, LLMs can be objective tools for sociological research in medicine.
  • Access is Not Enough: Because all patients in this study were already receiving care at a specialized academic epilepsy center, the observed disparities cannot be blamed on a lack of access to neurologists. This suggests that structural barriers (like the cost of meds or environmental stressors) persist even within the clinic walls.
  • Socioeconomics Trumps Race in Multivariable Models: While Black patients had worse outcomes in simple (univariable) models, this effect disappeared when income and insurance were added to the model. This highlights how structural racism in the US often manifests through economic channels, which then directly impact health outcomes.
  • The "Positive Control": The model correctly identified that patients on more medications were less likely to be seizure-free. This serves as a "sanity check," confirming the LLM was capturing the clinical reality of "refractory" (hard-to-treat) epilepsy.

Key Results

  • Algorithmic Fairness: No significant difference in LLM accuracy across demographics (p-values for bias tests were mostly > 0.80).
  • Gender Disparity: Female patients had significantly higher odds of recent seizures compared to men (OR 1.33, p = 3x10⁻⁸).
  • Socioeconomic Disparity: Patients with public insurance (OR 1.53) and those from the lowest-income zip codes (OR 1.33) were much less likely to be seizure-free.
  • Age Trends: Older patients (65+) were significantly more likely to be seizure-free than younger adults (OR 0.43).
  • Race: Black patients showed worse outcomes in univariable analysis, but in the multivariable analysis, the effect was not significant (OR 1.03, p = 0.66), suggesting socioeconomic factors were the primary drivers.

Limitations and Future Work

  • Data Granularity: The EHR used outdated binary frameworks for sex and gender, failing to capture the experiences of non-binary or transgender patients.
  • Single-Center Study: The data comes from one high-level US academic center; results might differ in community hospitals or different geographic regions.
  • The "Why" Question: While the LLM revealed that disparities exist, it cannot explain why. The authors suggest future work should investigate factors like medication adherence, systemic stress, and specific structural barriers to identify targets for intervention.
  • Internalized Bias: There is a theoretical risk that if the original human annotators were biased, the model could have learned that bias. However, the use of triplicate, blinded annotations makes this less likely.

Summary updated

Abstract

We found no evidence that our LLM was intrinsically biased against any demographic group. Seizure freedom extracted by LLM revealed disparities in seizure outcomes across several demographic groups. These findings highlight the critical need to reduce disparities in the care of people with epilepsy.
BibTeX
@article{xie2024disparities,
  author = {Kevin Xie and William K. S. Ojemann and Ryan S. Gallagher and Russell T. Shinohara and Alfredo Lucas and Chloe E. Hill and Roy H. Hamilton and Kevin B. Johnson and Dan Roth and Brian Litt and Colin A. Ellis},
  title = {{Disparities in seizure outcomes revealed by large language models}},
  journal = {J. Am. Medical Informatics Assoc.},
  year = {2024},
  doi = {10.1093/jamia/ocae047},
}

Loading related content...

Figure