Can collaborative learning be private, robust and scalable?

Dmitrii Usynin, Helena Klause, Daniel Rueckert, Georgios Kaissis

Abstract

Collaborative Machine Learning (CML) is pivotal for medical imaging, enabling multi-institutional model training without centralizing sensitive patient data. However, CML is vulnerable to privacy leaks and integrity attacks. We present PSREval, an open-source framework designed to navigate the trade-offs between privacy, robustness, and scalability. Our evaluation reveals that combining Differential Privacy (DP) with model quantization and adversarial training can harden models against attacks while maintaining utility, particularly in medical contexts like pneumonia prediction.

Introduction

In medical imaging, Collaborative Machine Learning (CML) allows for learning from diverse datasets while keeping data localized. Despite its promise, CML introduces significant security risks: adversaries may reconstruct private training data from model updates or "poison" the training process to induce misdiagnoses. While individual defenses like Differential Privacy (DP) for privacy, adversarial training for robustness, and quantization for scalability exist, their combined effects are poorly understood. This work seeks to reconcile these three pillars—privacy, robustness, and scalability—within a unified framework.

The PSREval Framework

To systematically investigate the interactions between these defensive mechanisms, we developed PSREval. This framework evaluates models across a pipeline that includes data preparation, adversarial training, DP-SGD implementation, and post-training quantization.

The PSREval framework pipeline: from data preparation and adversarial training to DP-SGD and post-training quantization.

The framework utilizes:

  • Differential Privacy (DP-SGD): Injecting noise during training to provide formal privacy guarantees using the Opacus library.
  • Model Compression: Static post-training quantization (converting 32-bit floating-point weights to 8-bit integers) to reduce communication overhead in federated settings.
  • Adversarial Training: Exposing models to malicious samples (e.g., PGD, FGSM) during training to build robustness.
  • Threat Modeling: Assessing both train-time poisoning and inference-time adversarial attacks (white-box and transfer attacks).

Key Insights and Results

Our experiments on CIFAR-10 and the Paediatric Pneumonia Prediction (PPPD) dataset yielded several critical insights into the "impossible trinity."

The Quantization Defense

Quantization proved to be more than a compression tool. By reducing weight precision, it limits the effective "perturbation budget" available to an attacker. In larger models, this resulted in an adversarial robustness increase of up to 20%. Quantization reduced model sizes by approximately 75% with negligible impact on accuracy (within ±1%).

DP as a Poisoning Shield

A major finding is the effectiveness of DP-SGD in mitigating training-time poisoning. Because DP limits the influence of any single training sample, it naturally suppresses the impact of malicious outliers.

DP-SGD significantly reduces the utility loss caused by adversarial data during training compared to vanilla training.

As shown in our results, even when an adversary controls up to 40% of the training data, DP-trained models maintain significantly higher utility compared to non-private "vanilla" models, which suffer severe performance degradation. While vanilla models see utility losses exceeding 25%, DP-SGD keeps the loss remarkably low, often under 5%.

Synergistic Defenses

We observed that adversarial training and DP can be synergistic. When combined, they achieve higher robustness with a lower utility penalty than when adversarial training is applied in isolation. However, we also identified a "DP Transfer Vulnerability": DP-trained models are particularly susceptible to adversarial samples generated by other models with the same architecture and DP training, suggesting that model transparency in private settings requires careful management.

Conclusion and Future Work

The PSREval framework demonstrates that the "impossible trinity" of CML can be balanced through a multi-layered defensive strategy. Future research will extend this evaluation to more complex medical tasks such as image segmentation and explore quantization-aware training to further optimize the trade-off between privacy, robustness, and performance.

Summary updated

Abstract

In federated learning for medical image analysis, the safety of the learning protocol is paramount. Such settings can often be compromised by adversaries that target either the private data used by the federation or the integrity of the model itself. This requires the medical imaging community to develop mechanisms to train collaborative models that are private and robust against adversarial data. In response to these challenges, we propose a practical open-source framework to study the effectiveness of combining differential privacy, model compression and adversarial training to improve the robustness of models against adversarial samples under train- and inference-time attacks. Using our framework, we achieve competitive model performance, a significant reduction in model's size and an improved empirical adversarial robustness without a severe performance degradation, critical in medical image analysis.
BibTeX
@inproceedings{usynin2022can,
  author = {Dmitrii Usynin and Helena Klause and Daniel Rueckert and Georgios Kaissis},
  title = {{Can collaborative learning be private, robust and scalable?}},
  booktitle = {DeCaF/FAIR @ MICCAI},
  year = {2022},
  url = {https://openreview.net/forum?id=dj8CpG9Xvb-},
}
Figure