Non-parametric regularization for class imbalance federated medical image classification

Jeffry Wicaksana, Zengqiang Yan, Kwang-Ting ChengarXiv 2024 · DOI · arXiv

Introduction: The "Double Whammy" of Medical Federated Learning

Federated Learning (FL) is a transformative approach for medical AI, enabling collaborative model training across institutions without compromising patient privacy. However, medical data is notoriously prone to severe class imbalance. This manifests as a "double whammy": local imbalance, where individual clinics rarely encounter specific rare conditions, and global heterogeneity, where disease distributions vary significantly between specialized centers and general hospitals.

Standard deep learning models typically rely on parametric layers (weights) to define class boundaries. In the presence of extreme imbalance, these weights become unstable, leading to "client drift" where local models diverge significantly from the global objective. This results in poor generalization, particularly for the minority classes that are often the most clinically significant.

Proposed Method: FedNPR

To address these challenges, the authors introduce FedNPR, a framework centered on Non-parametric Regularization (NPR). Instead of relying solely on a linear classifier's weights, FedNPR models the feature space directly using stable class prototypes.

Overview of the FedNPR framework, illustrating the interaction between the federated server and multiple hospitals with heterogeneous class distributions.

Non-parametric Regularization (NPR)

The core innovation is the use of prototypes as "anchors" in the feature space.

  1. Online Sub-clustering: Each client maintains $K$ sub-clusters (prototypes) per class. Using the Sinkhorn-Knopp algorithm, features are assigned to these clusters in real-time during training. This captures intra-class diversity (e.g., different visual presentations of the same disease) without "smearing" the representation.
  2. Feature Regularization: The NPR loss acts as a contrastive signal, pulling features toward their assigned class prototypes and pushing them away from prototypes of other classes. This stabilizes the feature extractor against the noise of imbalanced labels.
The NPR mechanism: (1) online clustering of features into K sub-centroids per class, followed by (2) feature regularization to stabilize the representation.

Additional Components

  • Balanced Softmax (BSM): To further mitigate classifier bias, BSM adjusts the model's output based on local class frequencies.
  • Personalization (FedNPR-Per): A variant where hospitals maintain local classifier heads while sharing a robust, collaboratively trained feature extractor.

Experimental Evaluation

The framework was evaluated on two critical medical benchmarks: Fed-ISIC2019 (skin lesion classification) and RSNA-ICH (intracranial hemorrhage detection).

Key Results

FedNPR consistently outperformed state-of-the-art (SOTA) federated methods:

  • Skin Lesion (ISIC): FedNPR-Per achieved 76.2% balanced accuracy (bACC), significantly higher than FedAvg (69.5%) and FedBABU (72.1%).
  • Brain Hemorrhage (ICH): FedNPR-Per reached 72.1% bACC, a >7% improvement over FedProx.
  • Plug-and-Play Utility: Adding the NPR module to existing algorithms like FedRep or FedBABU consistently yielded performance gains (up to 7% bACC), demonstrating its modularity.

Conclusion

FedNPR provides a robust solution to the instability caused by class imbalance in federated medical imaging. By leveraging non-parametric prototypes as stabilizers, the framework reduces client drift and ensures that rare but critical classes are accurately represented. Future work aims to extend this approach to semi-supervised settings, further reducing the reliance on large-scale labeled medical datasets.

Summary updated

Abstract

Limited training data and severe class imbalance pose significant challenges to developing clinically robust deep learning models. Federated learning (FL) addresses the former by enabling different medical clients to collaboratively train a deep model without sharing privacy-sensitive data. However, class imbalance worsens due to variation in inter-client class distribution. We propose federated learning with non-parametric regularization (FedNPR and FedNPR-Per, a personalized version of FedNPR) to regularize the feature extractor and enhance useful and discriminative signal in the feature space. Our extensive experiments show that FedNPR outperform the existing state-of-the art FL approaches in class imbalance skin lesion classification and intracranial hemorrhage identification. Additionally, the non-parametric regularization module consistently improves the performance of existing state-of-the-art FL approaches. We believe that NPR is a valuable tool in FL under clinical settings.
BibTeX
@misc{wicaksana2024non,
  author = {Jeffry Wicaksana and Zengqiang Yan and Kwang-Ting Cheng},
  title = {{Non-parametric regularization for class imbalance federated medical image classification}},
  year = {2024},
  eprint = {2407.12446},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.12446},
}
Figure