FedCM: Client Clustering and Migration in Federated Learning via Gradient Path Similarity and Update Direction Deviation

Peng Wang, Shoupeng Lu, Hao Yin, Banglie Yang, Tianli Zhu, Cheng DaiIJCAI 2025 · DOI

Abstract

Federated Learning (FL) faces significant challenges in real-world deployments due to non-identical data distributions across clients (statistical heterogeneity) and temporal changes in local data (distribution drift). Traditional FL approaches often fail to adapt to these dynamics, leading to suboptimal global models. We propose FedCM, a novel framework that addresses these issues through a multi-stage process: a warm-up phase to capture gradient trajectories, a graph-based clustering algorithm using a new "gradient path" similarity metric, and a dynamic migration strategy based on update direction deviation. Our evaluations on standard benchmarks demonstrate that FedCM significantly outperforms existing state-of-the-art methods in both accuracy and robustness to data drift.

Introduction

Federated Learning enables collaborative model training while preserving data privacy. However, two primary obstacles hinder its effectiveness:

  1. Statistical Heterogeneity: Diverse client data distributions (e.g., label skew) make a single global model insufficient for all participants.
  2. Data Distribution Drift: Client data patterns often evolve over time, rendering static clustering or global models obsolete.

Existing clustered federated learning (CFL) methods typically rely on static snapshots of model weights or require a predefined number of clusters, which limits their flexibility in dynamic environments. FedCM overcomes these limitations by analyzing the learning trajectory of clients and allowing for seamless migration between clusters as data drifts.

Methodology: The FedCM Framework

The FedCM framework consists of three integrated components designed to identify and maintain optimal client groupings throughout the training process.

Overview of the FedCM pipeline: 1) Warm-up training to collect gradient paths, 2) Personalized vector extraction, 3) Similarity matrix calculation, and 4) Adaptive cluster detection and migration.

1. Personalized Gradient Path Similarity

Rather than comparing static model weights, FedCM focuses on the trajectory of gradients during a warm-up phase. By tracking the sequence of gradient updates ($g_i^t$) in the classification layer, the system captures how a model learns from a specific client's data. The similarity between two clients $i$ and $j$ is calculated using a cosine similarity matrix $S_{ij}$ based on these gradient paths. This approach provides a more distinct "fingerprint" of client data than final weight values alone.

2. Adaptive Graph-Based Clustering

To avoid the need for pre-specifying the number of clusters (a common requirement in K-Means), FedCM employs an adaptive graph-based approach. Clients are represented as nodes in a graph, with edges weighted by their gradient path similarity. A modularity-optimization algorithm is then used to detect natural clusters within this graph, allowing the system to scale and adapt to the underlying data structure automatically.

3. Update Angle-Based Migration

To handle distribution drift, FedCM monitors the "angle" of each client's updates relative to its cluster's average. If a client's update direction begins to deviate significantly, it is flagged as an outlier. The system then "migrates" this client to a more compatible cluster. This mechanism ensures that the model remains performant even as user behavior changes, without requiring access to raw client data.

Experimental Evaluation

FedCM was rigorously tested on four computer vision benchmarks: CIFAR-10, CIFAR-100, Fashion-MNIST (FMNIST), and SVHN. These datasets were partitioned to simulate extreme label skews (pathological and standard).

Key Results

  • Accuracy Gains: On CIFAR-10 with pathological skew, FedCM achieved an accuracy of 90.94%, a 2% improvement over the state-of-the-art PACFL and a massive jump from the 38.30% achieved by standard FedAvg.
  • Resilience to Drift: In scenarios where data drift was introduced mid-training, FedCM maintained near-peak performance. While other methods saw performance drops of up to 30‰, FedCM's degradation was limited to a mere 1‰, demonstrating its ability to "heal" the system through rapid client migration.
  • Parameter-Free Efficiency: The adaptive clustering reached optimal performance without manual tuning of the cluster count, simplifying deployment in unknown environments.

Discussion and Insights

The success of FedCM highlights several critical insights for federated learning:

  • Gradients as Trajectories: The path taken during training is more informative than the destination. Gradient paths create sharper cluster boundaries, making it easier to distinguish between clients with different data distributions.
  • Classification Layer Focus: Focusing on the final classification layer is computationally efficient and acts as a strong proxy for task identity.
  • Privacy-Preserving Drift Detection: Using update angles allows for robust drift detection with minimal overhead and no additional privacy risks, as it relies on information already shared during the standard FL update process.

Conclusion and Future Work

FedCM provides a robust solution for the twin challenges of heterogeneity and drift in federated learning. By leveraging gradient path similarity and dynamic migration, it ensures that clients are always grouped with peers who share similar data patterns. Future research will explore the application of FedCM to non-classification tasks, such as regression and generative modeling, and further optimize the communication overhead associated with tracking gradient paths during the warm-up phase.

Summary updated

BibTeX
@inproceedings{wang2025fedcm,
  author = {Peng Wang and Shoupeng Lu and Hao Yin and Banglie Yang and Tianli Zhu and Cheng Dai},
  title = {{FedCM: Client Clustering and Migration in Federated Learning via Gradient Path Similarity and Update Direction Deviation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/706},
}
Figure