Abstract
Federated Learning (FL) enables collaborative model training without centralizing raw data, but it faces significant challenges in non-IID (non-Independent and Identically Distributed) environments where user data distributions vary widely. Standard global models often fail to provide optimal performance for individual clients. This paper presents Dual Calibration-based Personalized Federated Learning (DC-PFL), a framework designed to balance global knowledge transfer with local model personalization. DC-PFL introduces a two-way calibration mechanism: local feature extractors are calibrated using global class means, and the global classifier is refined using virtual samples generated from client-provided statistical summaries. Our evaluation on CIFAR-10 and CIFAR-100 benchmarks shows that DC-PFL significantly outperforms existing state-of-the-art methods, achieving an accuracy improvement of up to 9.22% on complex datasets.
Introduction
In Personalized Federated Learning (PFL), the objective is to develop models that are specifically tuned to each client's unique data while still benefiting from the collective intelligence of the entire network. Traditional approaches often partition models into a feature extractor and a classifier. However, these methods frequently encounter a "tug-of-war" between global and local objectives. Over-emphasizing global features can lead to a loss of local accuracy, while excessive focus on local data prevents clients from learning broader patterns. DC-PFL addresses this by establishing a robust communication protocol that preserves privacy while ensuring model compatibility across the network.
The DC-PFL Framework
The core innovation of DC-PFL lies in its Dual Calibration strategy, which treats the local model as a heterogeneous feature extractor and the global model as a shared, homogeneous classifier.
As shown in Figure 1, the process involves two primary steps:
- Local Feature Calibration: Clients receive global class means from the server. During local training, an auxiliary loss function encourages the local feature extractor to align its representations with these global averages. This ensures that the local "language" of features remains compatible with the global classifier.
- Global Classifier Calibration: To avoid sharing raw data or embeddings, clients transmit only the mean and covariance of their features per class. The server uses these statistics to generate "virtual samples" from Gaussian distributions, which are then used to fine-tune the global classifier. This allows the classifier to maintain performance on classes that may not be present in every client's local dataset.
Experimental Evaluation
We evaluated DC-PFL using the CIFAR-10 and CIFAR-100 datasets, modified to simulate extreme non-IID conditions. In our setup, each client in the CIFAR-10 experiments only had access to 2 out of 10 classes, while CIFAR-100 clients were limited to 10 out of 100 classes. We tested the framework across varying numbers of clients (10, 50, and 100) and different levels of client participation.
Key Results
DC-PFL demonstrated superior performance compared to eight state-of-the-art baselines, including FedAvg, FedProto, and FedGH.
As illustrated in Figure 6, DC-PFL consistently maintains higher accuracy as the complexity of the task (number of classes) increases. On the CIFAR-100 dataset, DC-PFL achieved a peak accuracy of 74.93%, representing a 9.22% improvement over the next best baseline. The method also proved robust to low client participation rates and heterogeneous model architectures, confirming its applicability to real-world federated systems.
Conclusion
DC-PFL introduces a privacy-preserving and effective mechanism for personalized federated learning. By leveraging statistical summaries to generate virtual data and aligning local features with global prototypes, it bridges the gap between specialist local models and generalist global knowledge. Future work will explore more sophisticated distribution modeling beyond the Gaussian assumption to capture even more complex data structures.
Reference: Tang, X., Yu, H., Tang, R., Ren, C., Li, A., & Li, X. (2024). Dual Calibration-based Personalised Federated Learning. Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI-24).
Summary updated