TL;DR
This paper introduces a novel federated network of 12 health systems that allows for the analysis of electronic health records (EHRs) related to COVID-19 without centralizing patient data. By harmonizing EHRs and using distributed algorithms, they can answer complex clinical questions and build predictive models, all while maintaining patient privacy and institutional control over data.
What Problem Does This Paper Address?
The COVID-19 pandemic highlighted an urgent need for rapid analysis of real-world patient data to inform clinical decision-making, understand disease progression, and evaluate treatment effectiveness. Traditional approaches often involve creating centralized registries, which face significant hurdles due to privacy concerns, data governance complexities, and the sheer volume of data. Furthermore, the evolving nature of the pandemic meant that the most valuable data elements were not always known or collected early on. Researchers and clinicians also struggle with understanding the structure and quality of data across different institutions, making it difficult to formulate and execute queries effectively. This paper addresses the challenge of enabling robust, privacy-preserving data discovery from distributed EHR systems to answer critical COVID-19 research questions.
Methods
The core of this work is a federated network approach. Instead of moving patient data, the analysis is performed locally at each participating health system. Here's a breakdown of their methodology:
- Distributed, Federated Network: 12 health systems, comprising 202 hospitals, harmonized their EHR data into the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). This standardization is crucial for interoperability.
- Question-Answering System: A public website (covid19questions.org) serves as the interface for users to submit questions in natural language.
- Consortium Hub and Lead/Responding Sites: A "Consortium Hub" triages questions, assesses feasibility, and assigns them to a "Lead Site." The Lead Site develops a standardized query (SQL) and concept sets. These are then distributed to "Responding Sites" who run the queries on their local data.
- Privacy-Preserving Aggregation: Only aggregate results (counts, statistics, regression coefficients) are shared with the Consortium Hub. Individual-level data never leaves the participating institutions.
- Federated Regression (GLORE): For more complex analyses like logistic regression, they employed a distributed algorithm (Grid Binary Logistic REgression - GLORE). This algorithm allows for the calculation of model parameters and variance-covariance matrices at each site, with only these aggregated results being shared and combined at the Consortium Hub. This ensures that no patient-level data is transferred.
- Iterative Refinement: The process involves iterative refinement of concept sets and SQL queries, with feedback loops between Lead and Responding Sites to ensure accuracy and address data quality issues.
Datasets
The study utilized de-identified electronic health record (EHR) data from 12 health systems, covering 202 hospitals across the US and one international academic medical center. The data was harmonized into the OMOP Common Data Model. The paper does not introduce a new benchmark dataset but rather leverages existing, distributed EHR data.
Key Insights and Ideas
- Privacy as a Foundation: The entire system is built around the principle of keeping patient data local, which significantly lowers privacy risks and facilitates broader participation from institutions with varying data governance policies.
- Federated Learning for Clinical Insights: Demonstrates the feasibility of applying federated learning techniques (like GLORE) to complex statistical modeling on distributed EHR data, achieving results comparable to centralized models without data transfer.
- Agility in Research: The question-answering framework allows for rapid responses to evolving research needs during a pandemic, enabling quick exploration of new hypotheses as understanding of the disease grows.
- Democratizing Data Access: By making queries, concept definitions, and aggregated results publicly available, the project promotes transparency and reproducibility, allowing other researchers to build upon their work.
- The Power of Standardization (OMOP CDM): The use of the OMOP CDM was critical for harmonizing data across diverse EHR systems, enabling consistent analysis.
Key Results
- The consortium successfully answered numerous clinical questions about COVID-19 using their federated system.
- They demonstrated that patients taking angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers had lower unadjusted in-hospital mortality rates.
- Multivariate regression analysis showed that while age, ethnicity, and sex were initially considered, only age was a significant predictor of mortality after adjustment.
- The federated regression model produced results comparable to local logistic regression models, validating the approach.
- The public website provided answers to various clinical questions, with a mechanism for users to submit new queries.
Limitations and Future Work
- Not Real-Time: The approach does not provide real-time answers due to the iterative nature of data processing, quality checks, and the need for inter-institution agreements.
- Geographic Bias: The initial consortium had a significant concentration of institutions in California, though efforts are underway to expand geographic diversity.
- ETL Dependency: The reliance on recurring ETL processes for OMOP CDM implementation presents a challenge for real-time data availability.
- Future Work: The authors suggest repurposing the network for other diseases, expanding participation, and integrating with other federated consortia for multivariate analyses. They also aim to automate more of the process to approach real-time capabilities.
Summary updated