TL;DR
This paper introduces CODA, an open-source platform designed for federated analysis and machine learning on distributed healthcare data. CODA addresses the challenges of data privacy and interoperability in multi-institutional healthcare research by enabling collaborative analysis without centralizing sensitive patient information, making advanced analytics more accessible.
What Problem Does This Paper Address?
Healthcare data is siloed across numerous institutions, making it difficult to conduct large-scale, multi-institutional research. Traditional approaches often involve pooling data, which raises significant privacy concerns, requires complex legal agreements, and is inflexible. The COVID-19 pandemic highlighted the limitations of this "pooled analysis" paradigm, demonstrating the need for a more agile and privacy-preserving method for knowledge exchange. Existing tools for federated analytics (FA) and federated learning (FL) often lack crucial features like built-in medical standards support, no-code visualization tools, explicit disclosure control, and comprehensive statistical computation capabilities, hindering their adoption in real-world healthcare settings.
Methods
The CODA platform is built on a microservice architecture, comprising:
- Site Nodes: Deployed within institutional firewalls, these nodes handle data ingestion, storage (in FHIR and DICOM formats), and local computation for FA/FL tasks. They ensure data is de-identified before processing.
- Orchestration Hub: Located outside institutional firewalls, this central component coordinates distributed computations without direct access to patient data. It collects aggregated statistics or model weights from site nodes.
- User Applications: A dashboard app for no-code queries and visualizations, and a notebook app (JupyterLab) for advanced analysis and ML model training.
Key methodological choices include:
- FHIR and DICOM Standards: Prioritizing these widely adopted healthcare data standards for interoperability.
- Microservice Architecture: Enhancing modularity, ease of updates, and collaborative development.
- Auditable Communication: Utilizing a single, auditable WebSocket channel between the hub and site nodes for enhanced oversight.
- No-Code and Advanced Tools: Offering both user-friendly visualization tools and a powerful notebook environment.
- Open-Source Philosophy: Releasing all code under the GNU General Public License to promote widespread adoption and collaboration.
Datasets
- MIMIC-IV: Used for demonstrating end-to-end multi-modal FL, specifically for predicting in-hospital mortality.
- MIMIC-CXR: Used in conjunction with MIMIC-IV to provide imaging data for the multi-modal FL demonstration.
The paper does not introduce new benchmarks but rather demonstrates the platform's capabilities on existing, well-known datasets.
Key Insights and Ideas
- Bridging the Gap for Non-Technical Users: The inclusion of no-code data visualization tools is a significant step towards making FA/FL accessible to a broader range of healthcare professionals, not just data scientists.
- Emphasis on Interoperability: The explicit choice to support FHIR and DICOM from the outset is crucial for real-world deployment, as it directly addresses a major bottleneck in existing systems.
- Practicality Over Cutting-Edge Privacy: While acknowledging the importance of advanced privacy techniques like differential privacy, the initial focus was on deployability and core functionality, with privacy controls aligned with REB-approved protocols for de-identified data. This pragmatic approach allows for faster adoption.
- Governance is Key: The development of a standardized Governance Framework is highlighted as essential for formalizing interinstitutional agreements and enabling collaborative research, addressing the organizational and legal hurdles.
Key Results
- Successful Deployment: The CODA platform was successfully deployed at 8 out of 9 participating hospitals in Québec, Canada, demonstrating its technical feasibility in a real-world, heterogeneous IT environment.
- Scalability: The platform successfully enrolled over 1 million patients, with over 46 million FHIR objects and 3.7 million imaging studies mapped.
- Federated Learning Performance: A multi-modality FL model trained on MIMIC-IV and MIMIC-CXR for in-hospital mortality prediction achieved an AUC of 0.82 ± 0.01, comparable to pooled analysis (0.81 ± 0.01), demonstrating the viability of FL for such tasks.
- Data Visualization: A real-time COVID-19 monitoring dashboard was deployed across 3 hospital sites, showcasing the platform's ability to provide actionable insights from distributed data.
Limitations and Future Work
- Single-Center Dataset for Initial Testing: The FL performance was initially demonstrated on the MIMIC-IV dataset, which originates from a single hospital. Future work will focus on assessing performance on more heterogeneous, multi-institutional datasets.
- No Formal Security Audit: The initial public release of the software has not undergone a formal security audit.
- Data Mapping Challenges: While CODA supports FHIR/DICOM, the feasibility study revealed that not all sites had the technical expertise to perform the required data mappings and de-identification, indicating a need for further support tools.
- Privacy Guarantees: While ethical considerations for de-identified data were addressed, advanced privacy-preserving algorithms (e.g., differential privacy) are not yet implemented but are a focus of ongoing work.
- Commercial Solutions: The study focused on non-proprietary solutions, and commercial alternatives for FL in medical data were not formally assessed.
Future work will involve enriching the cohort with more data sources, implementing more sophisticated FL algorithms, validating the platform on synthetic datasets, and developing tools to assist with data conversion to FHIR format.
Summary updated