Abstract
In clinical settings, medical image datasets are often decentralized across institutions and characterized by partial labeling and extreme class imbalance. Standard Federated Learning (FL) approaches typically fail to identify rare diseases—underrepresented classes (UCs)—as they prioritize common classes (CCs) shared across the network. This paper introduces FedFew, a three-stage federated framework designed to address these challenges. By integrating federated self-supervised learning, energy-based fine-tuning, and prototype-based inference, FedFew enables the detection of rare conditions with minimal supervision. Experimental results on the Chest-Xray14 dataset demonstrate that FedFew significantly outperforms traditional FL baselines, achieving high F1-scores on rare classes where standard models fail entirely.
1. Introduction
The deployment of deep learning models in healthcare is hindered by two primary obstacles: data privacy and the high cost of expert annotation. Privacy regulations, such as GDPR and HIPAA, prevent the aggregation of raw patient data into centralized repositories. While Federated Learning (FL) offers a solution by training models locally and aggregating only weights, it struggles with the inherent "noise" of clinical practice. Specifically, medical datasets are often partially labeled—where a specialist at one site may only annotate specific pathologies—and highly imbalanced, with rare diseases appearing in only a fraction of the total data.
Existing FL algorithms tend to overfit common classes (CCs) and treat underrepresented classes (UCs) as background noise or outliers. This paper presents FedFew, a novel framework that leverages self-supervision and energy-based modeling to bridge the gap between common and rare disease detection in a decentralized environment.
2. The FedFew Framework
FedFew decomposes the learning process into three distinct stages to maximize feature representation and classification accuracy for both common and rare conditions.
2.1 Stage 1: Federated Self-Supervised Learning (FSSL)
To establish a robust feature extractor without relying on sparse labels, the framework begins with Federated Self-Supervised Learning. Using the SimSiam architecture, the model learns general visual representations from all available images across the decentralized network. This stage ensures the model captures the fundamental characteristics of medical imagery (e.g., anatomical structures in chest X-rays) in a class-agnostic manner before any supervised fine-tuning occurs.
2.2 Stage 2: Energy-Based Fine-Tuning
The core innovation of FedFew lies in its use of Energy-Based Models (EBMs). During the fine-tuning phase on common classes, the model is trained to create a mathematical "energy gap." Images containing common pathologies are mapped to low-energy states, while images containing rare or unknown pathologies are pushed toward high-energy states. This effectively transforms the standard classifier into an anomaly detector capable of flagging images that do not fit the profile of the common classes.
2.3 Stage 3: Prototype-Based Inference
For the underrepresented classes, FedFew moves away from standard linear classification heads, which require large amounts of data to converge. Instead, it utilizes "prototypes"—the average feature vectors derived from the few available labeled examples of a rare disease. During inference, if the energy score from Stage 2 indicates a potential anomaly, the model compares the image's features to these global prototypes using nearest-neighbor matching (typically via Cosine Distance) to make a final prediction.
3. Experimental Evaluation
The researchers evaluated FedFew using the Chest-Xray14 benchmark dataset. To simulate a realistic federated ecosystem, the data was distributed across six simulated clients (hospitals).
- Class Distribution: 11 conditions (e.g., Infiltration, Effusion) were designated as Common Classes (CCs). Three conditions—Edema, Pneumonia, and Hernia—were treated as Underrepresented Classes (UCs), appearing in only a subset of clients with very few labels.
- Baselines: The framework was compared against standard multi-label classification (MLC) models trained via Federated Averaging (FedAvg).
4. Results and Discussion
The experimental results underscore the limitations of traditional FL and the efficacy of the FedFew approach.
- Detection of Rare Classes: Traditional MLC baselines failed completely on underrepresented classes, yielding an F1-score of 0.00. In contrast, FedFew achieved a breakthrough, notably reaching an F1-score of 0.70 for the Hernia class.
- Common Class Performance: The introduction of energy-based modeling did not degrade performance on common classes. In fact, the EBM approach served as a regularizer, improving the AUROC for common classes from 0.69 to 0.74.
- Privacy-Preserving Collaboration: By sharing prototypes (feature means) rather than raw images or complex gradients for rare classes, FedFew allows institutions to learn from rare cases seen elsewhere without compromising patient privacy.
5. Conclusion and Future Work
FedFew provides a viable path for training robust medical AI in the presence of data silos and label scarcity. By treating rare diseases as high-energy anomalies and utilizing prototype-based few-shot learning, the framework successfully identifies conditions that standard models ignore.
Future research will focus on automating the selection of energy thresholds, which currently requires manual tuning. Additionally, exploring more sophisticated distance metrics for prototype matching and scaling the framework to larger, more diverse federated networks remain key priorities for clinical deployment.
Summary updated