Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review

Mrinmay Sen, Shruti Aparna, Rohit Agarwal, Chalavadi Krishna MohanarXiv 2025 · DOI · arXiv

Introduction

Federated Learning (FL) has emerged as a cornerstone for privacy-preserving machine learning, enabling models to be trained across decentralized devices without raw data ever leaving the local source. However, a significant gap exists between theoretical FL models and real-world deployments. Most foundational algorithms, including the ubiquitous Federated Averaging (FedAvg), operate under the assumption that all clients are available for every communication round. In practice, this is rarely the case. This review examines the challenge of partial client participation and categorizes the state-of-the-art frameworks designed to maintain stability, efficiency, and fairness in distributed environments.

The Reality of Partial Participation

In real-world FL deployments—ranging from mobile smartphones to industrial IoT sensors—devices are subject to "Resource Heterogeneity." Factors such as fluctuating battery levels, intermittent network connectivity, and varying computational capabilities mean that only a fraction of the total client pool can contribute at any given time.

When clients frequently drop out or participate sporadically, several critical issues arise:

  1. Unstable Convergence: The global model may fail to converge or do so much slower than in full-participation scenarios.
  2. Model Drift: The global model can become biased toward the most active or "reliable" users, particularly when data is non-IID (Independent and Identically Distributed).
  3. Delayed Training: Systems may stall while waiting for updates from slow or disconnected clients.

A Taxonomy of Solutions

To address these gaps, researchers have developed a variety of technical strategies. These can be broadly categorized into five core approaches:

1. Selective Training

Frameworks like EmbracingFL mitigate the burden on resource-constrained devices by splitting the model. Instead of requiring every client to train the full architecture, "weak" clients may only train specific layers (typically the output-side), allowing them to remain in the training pool without exhausting their local resources.

2. Structured Client Participation

Moving away from simple random sampling, methods such as CyCP and FedAMD introduce intentionality into client selection. CyCP (Cyclic Client Participation) groups clients based on availability cycles (e.g., time zones), while FedAMD differentiates between "anchor" clients (large batches) and "miner" clients (small batches) to ensure a diverse representation of data across rounds.

3. Gradient Memory and History-Awareness

Methods like GradMA and MIFA leverage the server's ability to cache information. By storing the last known update from every client, the server can use cached updates as surrogates for clients that are currently offline. This maintains training momentum and prevents the global model from ignoring the data of sporadic participants.

4. Variance Reduction

Algorithms such as SCAFFOLD and FedVARP introduce control variates or surrogate updates to correct "client drift." These mechanisms ensure that even if the active subset of clients has a skewed data distribution, the global update is adjusted to reflect the broader population's characteristics.

5. Gradient Compression

To accommodate clients with limited bandwidth, frameworks like Fed-EF utilize error-feedback mechanisms and heavy compression. This allows devices to contribute meaningful updates without the communication overhead typically required by standard FL protocols.

Evaluation and Benchmarking

The effectiveness of these methods is typically evaluated using standard computer vision (MNIST, CIFAR-10/100, TinyImageNet) and natural language processing (Shakespeare, Sent140) benchmarks. A critical component of these evaluations is the use of Dirichlet distributions to simulate highly non-IID data, which tests the robustness of the algorithms against extreme data heterogeneity.

Key Performance Insights

  • Accuracy Gains: Variance reduction and memory-based methods consistently outperform standard FedAvg. In non-IID settings, SCAFFOLD has been shown to achieve approximately 80% accuracy, a significant improvement over FedAvg’s 72% on similar vision tasks.
  • The Power of Selection: Simply optimizing how clients are selected can yield high returns. CyCP has demonstrated a 5–10% accuracy improvement over random sampling by aligning training rounds with client availability cycles.
  • Efficiency Trade-offs: While Fed-EF achieves high communication efficiency (90-92% accuracy on non-IID data), it risks introducing bias. Conversely, FedAMD ensures high data diversity but at a much higher computational cost.

Challenges and Future Directions

Despite recent progress, several hurdles remain for the field:

  • Memory Overhead: History-aware methods (like MIFA) require the server to store updates for every client ($O(Nd)$ memory). This is often impractical for massive networks involving millions of devices or for Large Language Models (LLMs) with billions of parameters.
  • Synchronicity Issues: Many variance-reduction techniques assume synchronous communication, which is difficult to maintain in highly asynchronous edge networks.
  • Privacy and Security: Some server-aided methods rely on proxy datasets, which may conflict with the strict privacy mandates of FL.

Future research is trending toward unified frameworks that combine complementary techniques—such as merging memory-efficient clustering with momentum-based stabilization. Furthermore, there is a growing need for domain-aware strategies that move beyond computer vision to address the unique requirements of time-series and complex language tasks.

Summary updated

BibTeX
@misc{sen2025overcoming,
  author = {Mrinmay Sen and Shruti Aparna and Rohit Agarwal and Chalavadi Krishna Mohan},
  title = {{Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review}},
  year = {2025},
  eprint = {2506.02887},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.02887},
}
Figure