Abstract
This paper explores the intersection of privacy-preserving techniques, such as Differential Privacy (DP) and $k$-anonymity, and Byzantine-robust aggregation methods in Federated Learning (FL) for healthcare. The authors demonstrate that adaptive aggregation methods, specifically Adaptive Federated Averaging (AFA), can successfully detect and block malicious clients without sacrificing model accuracy, even when strict privacy-preserving noise is injected into the system.
Introduction
Federated Learning (FL) is highly attractive for healthcare because it allows multiple hospitals to collaboratively train deep learning models without sharing raw, sensitive patient data, ensuring compliance with regulations like HIPAA and GDPR. However, standard FL aggregation methods, such as Federated Averaging (FedAvg), are highly vulnerable to "poisoning" attacks from faulty or malicious clients.
While robust aggregation methods exist to filter out bad updates, and privacy methods exist to protect patient data, combining them presents a unique challenge. Privacy methods inherently inject noise or generalize data, which can confuse robust aggregation algorithms that rely on measuring the distance or similarity between client updates to spot anomalies. This paper investigates whether a system can be robust to bad actors while simultaneously providing mathematical guarantees for patient privacy.
Methods and Datasets
The authors evaluate three robust aggregation methods:
- Coordinate-wise Median (COMED): Resists outliers by taking the median of client parameters.
- Multi-Krum (MKRUM): Aggregates only the models closest to each other in Euclidean space.
- Adaptive Federated Averaging (AFA): Uses a Hidden Markov Model to track client behavior over time and dynamically block divergent clients.
These methods are combined with two privacy techniques:
- Differential Privacy (DP): Adds Laplacian noise to 10% of trained model parameters.
- $k$-anonymity: Generalizes quasi-identifying attributes so no record is distinguishable from at least $k-1$ others.
Experiments were conducted on two tabular healthcare datasets: the Pima Indians Diabetes Dataset (768 records, 10 clients) and the Cleveland Heart Disease Database (297 records, 5 clients).
Results and Discussion
A major takeaway is that privacy-preserving noise does not inherently destroy the ability of algorithms to detect malicious clients. The "signal" of a malicious attack often remains detectable despite the noise introduced by DP or $k$-anonymity.
In Experiment I (Diabetes), under DP with three bad clients, standard FedAvg diverged completely. MKRUM and AFA performed best, with AFA successfully detecting and blocking malicious clients when using $k$-anonymity.
In Experiment II (Heart Disease), which featured a smaller dataset, AFA and COMED achieved low error rates (~9%) despite attacks. However, MKRUM failed to learn effectively in this scenario. A critical finding was that $k$-anonymity caused all aggregation methods to fail on this smaller dataset, highlighting that syntactic privacy approaches require a critical mass of data to be effective.
Conclusion
The study confirms that adaptive robust aggregation, particularly AFA, can maintain model integrity in the presence of both malicious actors and privacy-preserving noise. While $k$-anonymity shows limitations on small datasets, Differential Privacy remains a robust option. Future work will explore hyperparameter optimization for AFA and its application to high-dimensional medical imaging tasks.
Summary updated