Knowledge abstraction and filtering based federated learning over heterogeneous data views in healthcare

Anshul Thakur, Soheila Molaei, Pafue Christy Nganjimi, Andrew A. S. Soltan, Patrick Schwab, Kim Branson, David A. CliftonNpj Digit. Medicine 2024 · OpenReview

TL;DR

This paper introduces a novel federated learning (FL) framework that tackles "data view heterogeneity" in healthcare data. By using a "knowledge abstraction and filtering" mechanism, it allows institutions to train a global model together without needing to manually align their diverse data formats, thus preserving privacy and maximizing data utility.

What Problem Does This Paper Address?

Healthcare data is incredibly valuable for training robust AI models, but privacy regulations prevent institutions from sharing it directly. Federated Learning (FL) offers a solution by training models collaboratively across institutions without data leaving its source. However, a major hurdle in FL, especially in healthcare, is "data view heterogeneity." This means that even when data is prepared for machine learning (ML-ready), its structure can vary wildly between institutions due to differences in how they record patient information, the services they offer, and their care standards. This heterogeneity can include:

  • Feature Disparity: Different institutions might have different sets of features (e.g., one has blood pressure, another doesn't).
  • Shuffled Data Views: The same features might be present, but in different orders within the data.
  • Varying Units/Scales: Measurements for the same feature (like bilirubin levels) might be in different units.

These inconsistencies make it impossible for a single global model to process data from all clients, forcing researchers to either perform laborious manual data alignment (which can lead to information loss) or abandon FL altogether. This paper aims to bridge this gap by enabling effective FL even with these diverse data views, without requiring manual preprocessing.

Methods

The core innovation of this paper is a knowledge abstraction and filtering mechanism. This mechanism works in two main parts:

  1. Knowledge Abstraction (Server-side): The server maintains a global "knowledge vector" (a learned representation). This vector acts as a shared, abstract understanding of the data across all clients.
  2. Knowledge Filtering (Client-side): Each client has a filtering module. This module takes the global knowledge vector and the client's local input data. It then "filters" or "abstracts" the knowledge vector based on the client's specific data, creating a unified, fixed-dimensional representation (an "embedding") that the global model can understand. This filtering is done locally and doesn't require the client to share its raw data.

This approach allows the global model to be trained on a consistent, shared latent space, regardless of the original data view heterogeneity at each client. The framework is designed to be "plug-and-play," meaning it can be integrated with existing FL algorithms like FedAvg and FedSGD.

Datasets

The authors evaluated their framework on three real-world healthcare datasets:

  • CURIAL: A tabular dataset from four UK National Health Service (NHS) Trusts, used for COVID-19 prediction. This dataset was also used to model data as graphs.
  • eICU Collaborative Research Database (eICU-CRD): A large, multi-center time-series dataset from 150 hospitals, used for predicting shock. For this study, 50 hospitals were selected.
  • MIMIC-III: Another extensive time-series dataset from a critical care unit, used for mortality prediction and phenotyping.

These datasets represent different data types (tabular, time-series, and graph) and varying numbers of clients, allowing for a comprehensive evaluation.

Key Insights and Ideas

  • Data View Heterogeneity is a Major FL Bottleneck: The paper clearly demonstrates that standard FL methods struggle significantly when data views differ, highlighting the need for specialized solutions.
  • "Knowledge Vector" as a Shared Context: The concept of a global knowledge vector acting as a shared context for all clients is a clever way to enable consistent learning without direct data sharing.
  • Client-Side Filtering Preserves Privacy: The filtering mechanism operates locally on each client, ensuring that raw, heterogeneous data never leaves the institution.
  • "Plug-and-Play" for Existing FL: The framework's compatibility with standard FL algorithms makes it easier to adopt and integrate into existing systems.
  • Robustness Across Data Types: The framework shows promise across tabular, time-series, and graph data, suggesting broad applicability in healthcare.

Key Results

The proposed framework consistently outperformed or matched baseline methods across various scenarios of data view heterogeneity (feature disparity, shuffled data, scaled data).

  • CURIAL Dataset: The proposed framework achieved the highest AUROC (0.897 ± 0.189) in the "Standard" scenario and consistently performed well across all heterogeneity scenarios, often outperforming standalone models and other FL baselines. For instance, in the "Feature Disparity" scenario, it achieved an AUROC of 0.894 ± 0.009, significantly better than FedAvg (0.852 ± 0.007).
  • eICU & MIMIC-III Datasets: Similar trends were observed, with the proposed framework showing strong performance, particularly in heterogeneity scenarios where other FL methods degraded. For example, on MIMIC-III in the "Feature Disparity" scenario, it achieved an AUROC of 0.821 ± 0.029, outperforming FedAvg (0.77 ± 0.04).
  • Information Sharing: The paper showed that the federated approach with knowledge filtering provided better performance than a standalone version of the filtering mechanism at each client, demonstrating effective information transfer.
  • Feature Utilization: In the "Feature Disparity" scenario, analysis of feature importance showed that the framework could effectively utilize additional features present in some clients but not others.

Limitations and Future Work

  • Privacy Aspects: While the framework is compatible with existing privacy-preserving techniques (like differential privacy), it doesn't inherently include them. Future work will focus on integrating these.
  • Statistical Heterogeneity: The current evaluation primarily focuses on data view heterogeneity. Future research will explore addressing statistical heterogeneity (differences in data distributions, patient demographics, etc.).
  • Free-Text Data: The framework currently focuses on structured data. Integrating free-text data (like clinical notes) is a significant future direction, as it presents unique challenges.
  • Knowledge Vector Dimensionality: The dimensionality of the knowledge vector was found to be a critical hyperparameter, requiring tuning for optimal performance.
  • Computational Complexity: While minimal, the framework does introduce a slight overhead for the knowledge vector and filtering module compared to standard FL.

Summary updated

Abstract

Robust data privacy regulations hinder the exchange of healthcare data among institutions, crucial for global insights and developing generalised clinical models. Federated learning (FL) is ideal for training global models using datasets from different institutions without compromising privacy. However, disparities in electronic healthcare records (EHRs) lead to inconsistencies in ML-ready data views, making FL challenging without extensive preprocessing and information loss. These differences arise from variations in services, care standards, and record-keeping practices. This paper addresses data view heterogeneity by introducing a knowledge abstraction and filtering-based FL framework that allows FL over heterogeneous data views without manual alignment or information loss. The knowledge abstraction and filtering mechanism maps raw input representations to a unified, semantically rich shared space for effective global model training. Experiments on three healthcare datasets demonstrate the framework’s effectiveness in overcoming data view heterogeneity and facilitating information sharing in a federated setup.
BibTeX
@inproceedings{thakur2024knowledge,
  author = {Anshul Thakur and Soheila Molaei and Pafue Christy Nganjimi and Andrew A. S. Soltan and Patrick Schwab and Kim Branson and David A. Clifton},
  title = {{Knowledge abstraction and filtering based federated learning over heterogeneous data views in healthcare}},
  booktitle = {Npj Digit. Medicine},
  year = {2024},
  url = {https://openreview.net/forum?id=xwpNwNvnTA},
}
Figure