A Disease Identification Algorithm for Medical Crowdfunding Campaigns: Validation Study

Steven S. Doerstling, Dennis Akrobetu, Matthew Engelhard, Felicia Chen, Peter A. UbelJournal of Medical Internet Research 2022 · DOI

Abstract

A disease identification algorithm combining pretrained natural language processing models and ICD-10-CM code-based disease categorization was able to detect 11 disease categories in medical crowdfunding campaigns with high precision and accuracy.

Introduction

Medical crowdfunding has emerged as a significant "shadow" healthcare safety net in the United States, with platforms like GoFundMe hosting billions of dollars in campaigns. For health policy researchers, these campaigns offer a unique window into medical debt and healthcare gaps. However, the data is largely unstructured, consisting of informal, conversational text. Previous analysis methods, such as manual review or simple keyword matching, fail to scale or capture the nuances of patient descriptions. This study introduces a robust Natural Language Processing (NLP) pipeline to accurately categorize these campaigns at scale.

Methodology

The researchers developed a hybrid NLP pipeline using Python and Spark NLP for Healthcare. The process involves two primary identification tracks to ensure comprehensive coverage of the unstructured text:

  1. Named Entity Recognition (NER): A pretrained model identifies medical diagnoses within the text, even when common misspellings occur (e.g., "lukemia" or "aneruism").
  2. Keyword Search: To capture implied diagnoses, the system searches for 11 specific treatments or procedures (e.g., "dialysis" or "chemotherapy") that act as proxies for disease categories.

Entities identified through these tracks are mapped to formal ICD-10-CM codes. These codes are then grouped into 11 broad disease chapters (e.g., Neoplasms, Cardiovascular diseases) using the Clinical Classifications Software Refined (CCSR).

Overview of the disease identification algorithm pipeline, illustrating the transition from raw campaign description to categorized disease terms.

Results

The algorithm was validated against a "ground truth" set of 400 expert-labeled campaigns. It achieved an overall accuracy of 95%, with a weighted average precision of 0.83 and an F1 score of 0.78.

The hybrid approach proved essential; the treatment-based keyword search identified over 3,000 campaigns missed by NER alone. For instance, the word search was responsible for identifying 25% of genitourinary disease cases. Performance was strongest in categories like Neoplasms (F1 = 0.96) and Genitourinary diseases (F1 = 0.88), while categories with more vague or infrequent descriptions, such as mental health, showed lower F1 scores (0.57).

Discussion and Conclusion

This study demonstrates that combining NER with procedure-based keyword searches creates a scalable, highly accurate tool for analyzing unstructured medical data. While limitations exist—such as difficulty distinguishing primary diagnoses from incidental mentions and the exclusion of certain conditions like pregnancy—this pipeline provides a vital resource for researchers. By automating the categorization of crowdfunding data, health policy experts can better understand the specific financial burdens and healthcare gaps facing patients in the U.S. healthcare landscape.

Summary updated

Abstract

A disease identification algorithm combining pretrained natural language processing models and ICD-10-CM code-based disease categorization was able to detect 11 disease categories in medical crowdfunding campaigns with high precision and accuracy.
BibTeX
@article{doerstling2022disease,
  author = {Steven S. Doerstling and Dennis Akrobetu and Matthew Engelhard and Felicia Chen and Peter A. Ubel},
  title = {{A Disease Identification Algorithm for Medical Crowdfunding Campaigns: Validation Study}},
  journal = {Journal of Medical Internet Research},
  year = {2022},
  doi = {10.2196/32867},
}
Figure