Dual Calibration-based Personalised Federated Learning

Xiaoli Tang, Han Yu, Run Tang, Chao Ren, Anran Li, Xiaoxiao Li

Abstract

Federated Learning (FL) enables collaborative model training without centralizing raw data, but it faces significant challenges in non-IID (non-Independent and Identically Distributed) environments where user data distributions vary widely. Standard global models often fail to provide optimal performance for individual clients. This paper presents Dual Calibration-based Personalized Federated Learning (DC-PFL), a framework designed to balance global knowledge transfer with local model personalization. DC-PFL introduces a two-way calibration mechanism: local feature extractors are calibrated using global class means, and the global classifier is refined using virtual samples generated from client-provided statistical summaries. Our evaluation on CIFAR-10 and CIFAR-100 benchmarks shows that DC-PFL significantly outperforms existing state-of-the-art methods, achieving an accuracy improvement of up to 9.22% on complex datasets.

Introduction

In Personalized Federated Learning (PFL), the objective is to develop models that are specifically tuned to each client's unique data while still benefiting from the collective intelligence of the entire network. Traditional approaches often partition models into a feature extractor and a classifier. However, these methods frequently encounter a "tug-of-war" between global and local objectives. Over-emphasizing global features can lead to a loss of local accuracy, while excessive focus on local data prevents clients from learning broader patterns. DC-PFL addresses this by establishing a robust communication protocol that preserves privacy while ensuring model compatibility across the network.

The DC-PFL Framework

The core innovation of DC-PFL lies in its Dual Calibration strategy, which treats the local model as a heterogeneous feature extractor and the global model as a shared, homogeneous classifier.

The DC-PFL architecture illustrating the dual calibration process between clients and the server.

As shown in Figure 1, the process involves two primary steps:

  1. Local Feature Calibration: Clients receive global class means from the server. During local training, an auxiliary loss function encourages the local feature extractor to align its representations with these global averages. This ensures that the local "language" of features remains compatible with the global classifier.
  2. Global Classifier Calibration: To avoid sharing raw data or embeddings, clients transmit only the mean and covariance of their features per class. The server uses these statistics to generate "virtual samples" from Gaussian distributions, which are then used to fine-tune the global classifier. This allows the classifier to maintain performance on classes that may not be present in every client's local dataset.

Experimental Evaluation

We evaluated DC-PFL using the CIFAR-10 and CIFAR-100 datasets, modified to simulate extreme non-IID conditions. In our setup, each client in the CIFAR-10 experiments only had access to 2 out of 10 classes, while CIFAR-100 clients were limited to 10 out of 100 classes. We tested the framework across varying numbers of clients (10, 50, and 100) and different levels of client participation.

Key Results

DC-PFL demonstrated superior performance compared to eight state-of-the-art baselines, including FedAvg, FedProto, and FedGH.

Performance comparison of DC-PFL against baselines on CIFAR-10 and CIFAR-100 across different numbers of classes per client.

As illustrated in Figure 6, DC-PFL consistently maintains higher accuracy as the complexity of the task (number of classes) increases. On the CIFAR-100 dataset, DC-PFL achieved a peak accuracy of 74.93%, representing a 9.22% improvement over the next best baseline. The method also proved robust to low client participation rates and heterogeneous model architectures, confirming its applicability to real-world federated systems.

Conclusion

DC-PFL introduces a privacy-preserving and effective mechanism for personalized federated learning. By leveraging statistical summaries to generate virtual data and aligning local features with global prototypes, it bridges the gap between specialist local models and generalist global knowledge. Future work will explore more sophisticated distribution modeling beyond the Gaussian assumption to capture even more complex data structures.

Reference: Tang, X., Yu, H., Tang, R., Ren, C., Li, A., & Li, X. (2024). Dual Calibration-based Personalised Federated Learning. Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI-24).

Summary updated

BibTeX
@inproceedings{tang2024dual,
  author = {Xiaoli Tang and Han Yu and Run Tang and Chao Ren and Anran Li and Xiaoxiao Li},
  title = {{Dual Calibration-based Personalised Federated Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=hvGlcmTQ7l},
}
Figure