Combining Federated Machine Learning and Qualitative Methods to Investigate Novel Pediatric Asthma Subtypes: Protocol for a Mixed Methods Study

Jie Xu, Sankalp Talankar, Jinqian Pan, Ira Harmon, Yonghui Wu, David A Fedele, Jennifer Brailsford, Jennifer Noel FisheJMIR Research Protocols 2024 · OpenReview

TL;DR

This paper proposes a novel mixed-methods approach to identify and characterize pediatric asthma subtypes using federated machine learning on real-world electronic health record (EHR) data. By combining privacy-preserving ML with qualitative clinician feedback, the study aims to develop clinically actionable insights for personalized asthma care.

What Problem Does This Paper Address?

Pediatric asthma is a complex disease with significant heterogeneity, meaning it manifests differently across children. Current classifications of asthma subtypes are limited, often relying on simple dichotomies like "allergic vs. non-allergic." This oversimplification fails to capture the nuances of disease progression, severity, and response to treatment, hindering the development of personalized care strategies. Analyzing large, longitudinal EHR datasets holds promise for uncovering more granular subtypes, but the sensitive nature of Protected Health Information (PHI) presents a major privacy hurdle for traditional centralized ML approaches.

Methods

The study employs a multi-pronged approach:

  • Data Preprocessing and Feature Engineering:
    • Computable Phenotype (CP) Development: The researchers will develop and refine a "research-grade" pediatric asthma CP. This involves creating algorithms to accurately identify pediatric asthma patients (aged 2-18) from EHR data, going beyond simple ICD codes by incorporating structured data and free-text clinical notes. They plan to leverage existing CPs and, if necessary, develop new ones using ML models, including transformer-based architectures.
  • Clinical Natural Language Processing (NLP) Pipeline: To extract valuable information from unstructured clinical notes (e.g., symptoms, treatment adherence), they will develop a clinical NLP pipeline. This will involve systematic annotation of notes and the application of advanced NLP models, such as GatorTron, to handle negation and abbreviations.
  • Federated Machine Learning for Subtyping:
    • Patient Representation Learning: Deep learning models (e.g., LSTMs, transformers) will be used to create temporal representations of patient data, capturing disease progression over time in 3-month blocks.
  • Federated Clustering: To identify subtypes without centralizing sensitive data, federated learning will be employed. This allows local sites to train models on their data, share model parameters (not raw data) with a central server, and collaboratively build a global model. Hierarchical agglomerative clustering (Ward method) will then be used to group patients into distinct temporal progression subtypes.
  • Qualitative Implementation Science:
    • Focus Groups: To assess the clinical utility and interpretability of the identified subtypes, sequential rounds of focus groups will be conducted with practicing pediatric asthma clinicians.
  • EHR Prototype Development: Based on clinician feedback and a user-centered design approach, prototypes for visualizing subtype information within EHR systems will be created. A second round of focus groups will evaluate these prototypes.
  • Framework: The Promoting Action on Research Implementation in Health Services (PARIHS) framework will guide the qualitative research to ensure findings are translatable into clinical practice.

Datasets

  • OneFlorida+ Clinical Research Network: This is the primary data source, comprising linked and longitudinal patient-level real-world data (RWD) from over 20 million patients across Florida, Georgia, and Alabama. The dataset includes demographic information, diagnoses, procedures, vital signs, medications, laboratory results, vital statistics, and EHR data from 15 clinical partners. Data spans from 2011 to 2023.
  • Specifics: As of September 2024, the dataset contains data for 411,628 patients aged 2-18 years and 11,156,148 clinical notes.

Key Insights and Ideas

  • Privacy-Preserving Subtyping is Crucial: The paper highlights the critical need for privacy-preserving methods like federated learning when working with sensitive EHR data for ML-driven subtype discovery. This approach enables collaboration across institutions without compromising patient confidentiality.
  • Beyond Static Subtypes: Temporal Progression Matters: The study emphasizes that asthma is a dynamic disease. Identifying subtypes based on temporal progression patterns (how a patient's disease state evolves over time) is more informative than static classifications and can lead to better predictions and interventions.
  • Bridging the Gap Between ML and Clinical Practice: The integration of qualitative methods (focus groups) and user-centered design for EHR prototypes is a key strength. This ensures that the ML-derived subtypes are not just statistically significant but also clinically relevant and usable by healthcare providers.
  • Leveraging Diverse Data Sources: The plan to incorporate structured EHR data, unstructured clinical notes (via NLP), and potentially social determinants of health (SDOH) data promises a more comprehensive understanding of asthma heterogeneity.

Key Results

This paper is a protocol, so it outlines the planned methodology and expected outcomes rather than presenting finalized results. However, the expected outcomes are:

  • Development of a research-grade pediatric asthma CP and NLP pipeline.
  • Identification of novel pediatric asthma subtypes and their temporal progression pathways using federated learning.
  • Assessment of the clinical utility of these subtypes through clinician focus groups.
  • Creation of EHR prototypes for visualizing subtype information.
  • Publication of subtyping results in year 4 and EHR prototype work at the end of year 5.

Limitations and Future Work

  • Generalizability: The study acknowledges that the OneFlorida+ network, while large, may not be fully representative of all pediatric asthma patients nationwide or worldwide. Therefore, the identified subtypes may have limited generalizability.
  • Missing Data: Missing data is a potential challenge, particularly for patient-reported variables. The researchers plan to address this using imputation techniques.
  • Future Work: The protocol outlines a clear 5-year plan, with future work focusing on the implementation and dissemination of findings, including the development of EHR-based clinical decision support systems. The federated learning framework is also suggested for application to other chronic diseases.

Summary updated

Abstract

Background: Pediatric asthma is a heterogeneous disease; however, current characterizations of its subtypes are limited. Machine learning (ML) methods are well-suited for identifying subtypes. In particular, deep neural networks can learn patient representations by leveraging longitudinal information captured in electronic health records (EHRs) while considering future outcomes. However, the traditional approach for subtype analysis requires large amounts of EHR data, which may contain protected health information causing potential concerns regarding patient privacy. Federated learning is the key technology to address privacy concerns while preserving the accuracy and performance of ML algorithms. Federated learning could enable multisite development and implementation of ML algorithms to facilitate the translation of artificial intelligence into clinical practice.Objective: The aim of this study is to develop a research protocol for implementation of federated ML across a large clinical research network to identify and discover pediatric asthma subtypes and their progression over time.Methods: This mixed methods study uses data and clinicians from the OneFlorida+ clinical research network, which is a large regional network covering linked and longitudinal patient-level real-world data (RWD) of over 20 million patients from Florida, Georgia, and Alabama in the United States. To characterize the subtypes, we will use OneFlorida+ data from 2011 to 2023 and develop a research-grade pediatric asthma computable phenotype and clinical natural language processing pipeline to identify pediatric patients with asthma aged 2-18 years. We will then apply federated learning to characterize pediatric asthma subtypes and their temporal progression. Using the Promoting Action on Research Implementation in Health Services framework, we will conduct focus groups with practicing pediatric asthma clinicians within the OneFlorida+ network to investigate the clinical utility of the subtypes. With a user-centered design, we will create prototypes to visualize the subtypes in the EHR to best assist with the clinical management of children with asthma.Results: OneFlorida+ data from 2011 to 2023 have been collected for 411,628 patients aged 2-18 years along with 11,156,148 clinical notes. We expect to complete the computable phenotyping within the first year of the project, followed by subtyping during the second and third years, and then will perform the focus groups and establish the user-centered design in the fourth and fifth years of the project.Conclusions: Pediatric asthma subtypes incorporating RWD from diverse populations could improve patient outcomes by moving the field closer to precision pediatric asthma care. Our privacy-preserving federated learning methodology and qualitative implementation work will address several challenges of applying ML to large, multicenter RWD data.International Registered Report Identifier (IRRID): DERR1-10.2196/57981JMIR Res Protoc 2024;13:e57981doi:10.2196/57981
BibTeX
@inproceedings{xu2024combining,
  author = {Jie Xu and Sankalp Talankar and Jinqian Pan and Ira Harmon and Yonghui Wu and David A Fedele and Jennifer Brailsford and Jennifer Noel Fishe},
  title = {{Combining Federated Machine Learning and Qualitative Methods to Investigate Novel Pediatric Asthma Subtypes: Protocol for a Mixed Methods Study}},
  booktitle = {JMIR Research Protocols},
  year = {2024},
  url = {https://openreview.net/forum?id=OY1HpM1lig},
}
Figure