Challenges in Implementing the Local Node Infrastructure for a National Federated Machine Learning Network in Radiology

Paul Jacobs, Constantin Ehrengut, Andreas Bucher, Tobias Penzkofer, Mathias Lukas, Jens Kleesiek, Timm DeneckeHealthcare 2023 · DOI

TL;DR

This paper tackles the practical challenges of setting up local infrastructure for national federated machine learning (FL) networks in radiology. It proposes essential requirements for data governance, workflows, and local node setup, drawing on the German RACOON initiative, and recommends integrating FL nodes into existing hospital IT for smoother implementation and maintenance.

What Problem Does This Paper Address?

Developing robust machine learning models for medical imaging, particularly radiology, requires vast, diverse datasets. However, aggregating such large datasets from multiple institutions poses significant challenges related to data privacy, security, and logistical complexity. Federated learning offers a solution by enabling model training on decentralized data without direct data sharing. Yet, implementing FL requires a sophisticated infrastructure at each participating site (local node) to handle data governance, processing, and secure communication. This paper addresses the practical hurdles and provides guidance on establishing these local nodes, using the RACOON network as a case study.

Methods

The authors propose a framework for local node infrastructure, focusing on three key areas:

  • Data Governance and Data Sharing: Emphasizing the critical need for robust policies on data access, storage, and sharing. They highlight the importance of anonymization and pseudonymization techniques, advocating for strong pseudonymization where re-identification keys remain local. The paper also touches upon the role of Joint Controllership Agreements (JCAs).
  • Data Science Workflow and Data Processing: Outlining a typical workflow involving data retrieval from local PACS/HIS, pre-processing (normalization, annotation, segmentation), model implementation, training, testing, and inference. They suggest a modular approach with three building blocks: Data Integration, Pre-processing, and Modelling.
  • Local Node Infrastructure Requirements: Detailing the necessary hardware (processing power, GPU, memory, storage), reliable network connections, and security measures (access controls, encryption). They discuss the use of virtual machines (VMs) and containers for creating isolated and standardized environments.

A significant methodological contribution is the comparison of two deployment strategies:

  1. Integrated Installation: Integrating the local node directly into the existing hospital IT infrastructure.
  2. Separate Installation: Setting up the node in a dedicated, subordinated environment within the radiology department.

The paper analyzes the advantages and disadvantages of each approach, particularly in terms of flexibility, administrative access, data security, configuration effort, hardware integration, and maintenance.

Datasets

The paper does not introduce new datasets or benchmarks. Instead, it discusses the use of existing medical imaging data formats like DICOM and NIfTI, and the HL7 FHIR standard for data exchange. The RACOON project itself is presented as a large-scale multicentric network that has aggregated over 16,000 Thorax CTs and 14,000 curated datasets with structured reports and segmentations.

Key Insights and Ideas

  • Federated Learning is Promising, but Implementation is Complex: While FL offers a powerful way to leverage medical data for AI without compromising privacy, the practicalities of setting up and maintaining local nodes are substantial.
  • Data Governance is Paramount: Establishing clear, legally compliant data governance policies is the foundational step, and differences in regulatory laws can significantly delay projects.
  • Integrated vs. Separate Node Installation: Integrating the local node into existing hospital IT offers benefits in maintenance and deployment effort due to pre-existing infrastructure and backup strategies. However, it can lead to less flexibility and potential restrictions from IT departments. A separate installation provides more flexibility but requires building infrastructure from scratch.
  • VM Conversion Challenges: The paper highlights a real-world pitfall where converting VM images (e.g., from Hyper-V to VMware) can cause delays and integration issues, underscoring the importance of compatibility.
  • RACOON as a Blueprint: The RACOON initiative serves as a valuable use case, demonstrating how a national FL network can be built and the lessons learned from its implementation.

Key Results

The paper presents qualitative findings based on the RACOON project experience:

  • Successful Network Growth: The RACOON network has successfully grown to include 38 local nodes, contributing a substantial dataset for research.
  • Advantages of Integrated Installation: The authors found that integrating the node into the existing clinical IT infrastructure provided benefits like a robust daily VM snapshot backup strategy, minimizing data loss.
  • Challenges Encountered: Difficulties included negotiating data sharing agreements across different jurisdictions, hardware integration limitations (e.g., single GPU capacity), and VM format conversion issues.
  • Recommendation: The authors recommend integrating the local node into existing clinical IT infrastructure as the preferred approach, provided certain specifications can be met, due to its efficiency in maintenance and deployment.

Limitations and Future Work

  • Site-Specific Realization: The authors acknowledge that the exact administrative and technical structure of local nodes will vary significantly between sites, making a universally applicable "one-size-fits-all" solution difficult.
  • Future Work: They suggest that future research should focus on developing an easily deployable and generic framework to simplify the implementation of local nodes for federated learning networks. This would help ensure congruence with other local nodes and the overall network, guaranteeing a functioning interface for data transfer, updates, and integration.

Summary updated

Abstract

Data-driven machine learning in medical research and diagnostics needs large-scale datasets curated by clinical experts. The generation of large datasets can be challenging in terms of resource consumption and time effort, while generalizability and validation of the developed models significantly benefit from variety in data sources. Training algorithms on smaller decentralized datasets through federated learning can reduce effort, but require the implementation of a specific and ambitious infrastructure to share data, algorithms and computing time. Additionally, it offers the opportunity of maintaining and keeping the data locally. Thus, data safety issues can be avoided because patient data must not be shared. Machine learning models are trained on local data by sharing the model and through an established network. In addition to commercial applications, there are also numerous academic and customized implementations of network infrastructures available. The configuration of these networks primarily differs, yet adheres to a standard framework composed of fundamental components. In this technical note, we propose basic infrastructure requirements for data governance, data science workflows, and local node set-up, and report on the advantages and experienced pitfalls in implementing the local infrastructure with the German Radiological Cooperative Network initiative as the use case example. We show how the infrastructure can be built upon some base components to reflect the needs of a federated learning network and how they can be implemented considering both local and global network requirements. After analyzing the deployment process in different settings and scenarios, we recommend integrating the local node into an existing clinical IT infrastructure. This approach offers benefits in terms of maintenance and deployment effort compared to external integration in a separate environment (e.g., the radiology department). This proposed groundwork can be taken as an exemplary development guideline for future applications of federated learning networks in clinical and scientific environments.
BibTeX
@inproceedings{jacobs2023challenges,
  author = {Paul Jacobs and Constantin Ehrengut and Andreas Bucher and Tobias Penzkofer and Mathias Lukas and Jens Kleesiek and Timm Denecke},
  title = {{Challenges in Implementing the Local Node Infrastructure for a National Federated Machine Learning Network in Radiology}},
  booktitle = {Healthcare},
  year = {2023},
  doi = {10.3390/healthcare11172377},
}
Figure