FeARH: Federated machine learning with anonymous random hybridization on electronic medical records

Jianfei Cui, He Zhu, Hao Deng, Ziwei Chen, Dianbo LiuJ. Biomed. Informatics 2021 · OpenReview

TL;DR

This paper introduces FeARH, a novel federated learning approach for electronic medical records that enhances privacy by anonymously hybridizing model parameters. By introducing randomization during parameter exchange, FeARH aims to protect sensitive patient data even when central analyzers are untrustworthy, achieving performance comparable to traditional centralized and federated learning methods.

What Problem Does This Paper Address?

Training machine learning models on electronic medical records (EMRs) is highly valuable for advancing healthcare, but it's severely hampered by privacy regulations and the distributed nature of EMR data. Centralizing this data for traditional machine learning is often infeasible due to privacy concerns and logistical challenges. Federated learning offers a solution by training models across distributed data silos without centralizing raw data. However, existing federated learning methods often rely on a trusted central analyzer to aggregate model updates. This paper addresses the critical vulnerability where this central analyzer might be untrustworthy, potentially compromising patient privacy. The authors propose a method to mitigate risks even when the central party is malicious or compromised.

Methods

The core innovation of this paper is the Federated machine learning with Anonymous Random Hybridization (FeARH) algorithm. Unlike traditional federated learning where local models are directly sent to a central server for aggregation, FeARH introduces an intermediate "hybridization" step.

Here's how it works:

  1. Local Training: Each data owner trains their local model on their private data.
  2. Parameter Hybridization: Before sending models to the central analyzer, local models exchange a randomly selected subset of their parameters with each other. This exchange is randomized, meaning not all parameters are shared, and the selection is random. This process is designed to "degenerate the integration of connections between medical record data and models' parameters."
  3. Anonymous Aggregation: The hybridized (and thus partially randomized) model parameters are then sent to the central analyzer. The analyzer aggregates these parameters to form a global federated model.
  4. Iterative Refinement: This process (local training, hybridization, aggregation) is repeated iteratively until a halting condition (e.g., minimal improvement in AUCROC) is met.

The key novelty lies in the anonymous random hybridization, which aims to prevent a malicious central analyzer from reconstructing sensitive information by making it difficult to trace specific model parameters back to individual data owners or their data.

Datasets

The study utilizes the eICU Collaborative Research Database. This is a large, multi-center critical care database made available by Philips Healthcare and the MIT Laboratory for Computational Physiology. The dataset contains information on 30,760 patients, including medicines taken and mortality. For their experiments, the authors used this data to train neural network models to predict mortality based on medication intake. The dataset was split into training (70%), testing (20%), and validation (10%) sets. For simulation purposes, the data was further divided into 8 sub-datasets, representing 8 data owners.

Key Insights and Ideas

  • Privacy through Randomization: The core insight is that introducing controlled randomness into the parameter exchange process can significantly enhance privacy in federated learning, even against untrustworthy central entities.
  • Decoupling Data from Parameters: FeARH effectively decouples the model parameters from the specific data that generated them. By hybridizing and randomizing, it becomes much harder for an attacker to infer specific patient data from the aggregated model.
  • Balancing Performance and Privacy: The paper demonstrates that this enhanced privacy doesn't necessarily come at a steep performance cost. FeARH achieves comparable results to centralized and standard federated learning, suggesting a viable trade-off.
  • "Active" Adversary Mitigation: The method is specifically designed to counter an "active" adversary, meaning one that can not only observe but also potentially manipulate the training process or model parameters.

Key Results

The paper compares the performance of three methods: Centralized Learning, Original Federated Learning, and FeARH, using AUCROC and AUCPR as evaluation metrics.

  • Neural Network Performance:
    • Centralized Learning: AUCROC of 0.8412, AUCPR of 0.6794.
  • Original Federated Learning: AUCROC of 0.8452, AUCPR of 0.7019.
  • FeARH: AUCROC of 0.8312, AUCPR of 0.6778. While FeARH showed slightly lower scores than the other two methods, the performance was comparable, especially considering the enhanced privacy guarantees.

  • Logistic Regression Performance:

    • Centralized Learning: AUCROC of 0.8412, AUCPR of 0.4579.
  • Original Federated Learning: AUCROC of 0.7816, AUCPR of 0.5989.
  • FeARH: AUCROC of 0.7864, AUCPR of 0.6206. Interestingly, for logistic regression, FeARH outperformed original federated learning in AUCPR.

  • Privacy Preservation: An additional experiment using a Generative Adversarial Network (GAN) as an attack model showed that the attacker achieved significantly lower accuracy (0.6919 vs. 0.9957) and higher loss (3.0011 vs. 0.0976) when attacking FeARH compared to original federated learning. This indicates FeARH is more resistant to information leakage.

  • Hybrid Exchange Rate Impact: The study found that increasing the hybrid exchange rate (γ) generally led to a decrease in AUCROC and AUCPR. This highlights a direct trade-off: higher exchange rates offer better privacy but may slightly reduce model performance.

Limitations and Future Work

  • Performance Trade-off: The paper acknowledges that FeARH's AUCROC and AUCPR scores are slightly lower than centralized and original federated learning in some cases, indicating a potential performance-privacy trade-off. The optimal hybrid exchange rate (γ) needs to be carefully chosen based on the specific application's privacy and performance requirements.
  • Computational Overhead: While not explicitly detailed, hybridization likely introduces some additional computational overhead compared to direct aggregation in traditional federated learning.
  • Generalizability: The experiments were conducted on a specific medical dataset. Future work could explore the effectiveness of FeARH on a wider range of datasets and machine learning models beyond neural networks and logistic regression.
  • Parameter Selection: The choice of the hybrid exchange rate (γ) is presented as a subjective parameter. Further research could explore more automated or adaptive methods for determining optimal exchange rates.
  • Exploring Other Models: The authors mention that the analysis of other models is beyond the scope of this paper, suggesting future work could extend FeARH to other machine learning architectures.

Summary updated

Abstract

Highlights•A new algorithm is proposed, federated machine learning with anonymous random hybridization (FeARH), to deal with untrustworthy situations.•Hybridizing parameters to degenrate connection between machine learning models’parameters and private medical information before sharing with other institutions.•Reaching comparable result as centralized machine learning and original federated machine learning.
BibTeX
@article{cui2021fearh,
  author = {Jianfei Cui and He Zhu and Hao Deng and Ziwei Chen and Dianbo Liu},
  title = {{FeARH: Federated machine learning with anonymous random hybridization on electronic medical records}},
  journal = {J. Biomed. Informatics},
  year = {2021},
  url = {https://openreview.net/forum?id=e0hMKGr6IG},
}
Figure