Client Selection in Federated Learning: Principles, Challenges, and Opportunities

Lei Fu, Huanle Zhang, Ge Gao, Huajie Wang, Mi Zhang, Xin Liu

Abstract

As a privacy-preserving paradigm for training Machine Learning (ML) models, Federated Learning (FL) has received tremendous attention from both industry and academia. In a typical FL scenario, clients exhibit significant heterogeneity in terms of data distribution and hardware configurations. Thus, randomly sampling clients in each training round may not fully exploit the local updates from heterogeneous clients, resulting in lower model accuracy, slower convergence rate, degraded fairness, etc. To tackle the FL client heterogeneity problem, various client selection algorithms have been developed, showing promising performance improvement. In this paper, we systematically present recent advances in the emerging field of FL client selection and its challenges and research opportunities.

Introduction

Federated Learning (FL) has evolved from a specialized privacy-preserving technique into a fundamental architecture for modern machine learning. It powers diverse applications, from predictive text on smartphones to collaborative analytics in healthcare, by allowing models to be trained on decentralized data. However, the decentralized nature of FL presents a significant challenge: the "wild west" of client devices. Unlike controlled data center environments, FL must contend with a heterogeneous pool of clients that vary wildly in hardware performance and data characteristics.

The standard Federated Learning (FL) training cycle involving parameter download, local training, parameter upload, and aggregation.

As illustrated in Figure 1, the typical FL process involves a central server and multiple clients. The server distributes the current model parameters (1), clients perform local training (2) on their private data, and then upload their updated parameters (3) back to the server for aggregation (4). The efficiency of this iterative loop is highly sensitive to the subset of clients selected in each round.

The Problem: Heterogeneity and Random Sampling

In ideal theoretical models, a central server picks a random subset of clients. In practice, this approach is often inefficient due to two main types of heterogeneity:

  1. System Heterogeneity: Client devices range from high-end smartphones with dedicated NPUs to older, resource-constrained budget devices. If a server waits for all selected clients to finish, the slowest "stragglers" dictate the pace of the entire training round, leading to significant time waste.
  2. Statistical Heterogeneity: Real-world data is rarely Independent and Identically Distributed (IID). For example, different users have unique linguistic patterns or photo-taking habits. Randomly picking a biased sample can cause the global model's accuracy to plummet or result in "unfair" models that perform poorly for specific user groups.

The central question for researchers is: How do we intelligently pick the "right" clients in each round to maximize model performance while minimizing resource waste?

The Utility Framework for Client Selection

Most advanced client selection algorithms move beyond random sampling by employing a Utility Function. This framework assigns a score to each client based on their potential contribution to the global model: $$Util(i) = Util_{stat}(i) \times Util_{sys}(i)$$

1. Statistical Utility (Data Quality)

This measures how much a client's local update will improve the global model. Algorithms may use data-based metrics, such as the number of local samples or the training loss, where high loss often indicates "challenging" data that the model needs to learn. Alternatively, model-based metrics compare local weight updates to the global model to identify unique or valuable information.

2. System Utility (Hardware Efficiency)

This evaluates a client's ability to complete training within a required timeframe. Algorithms often implement deadlines; if a client is too slow, they may be excluded or their contribution penalized to maintain the training velocity.

A summary of existing client selection algorithms and their focus on system and data heterogeneity.

As shown in Figure 2, various research works have tackled these dimensions. For instance, FedCS focuses on maximizing participation within deadlines, while frameworks like Oort and PyramidFL jointly exploit both data and system heterogeneity to optimize the overall "wall-clock" convergence time.

Datasets and Evaluation

To test these selection strategies, researchers use two main types of datasets:

  • Synthetic Partitions: Standard datasets like MNIST or CIFAR-10 are artificially split to simulate non-IID data.
  • Realistic Partitions: Datasets like Reddit or StackOverflow, which naturally include user IDs, are preferred as they capture authentic human behavior and device availability patterns.

Key Insights and Future Directions

Synthesized findings from the field highlight the high stakes of client selection. Ignoring heterogeneity can reduce model accuracy by up to 9.2% and slow down convergence by over 2.6x.

Several critical insights have emerged:

  • The "Later-is-Better" Phenomenon: Some research suggests increasing the number of clients per round as training progresses to capture finer model details.
  • Communication Bottlenecks: In many mobile scenarios, the time taken to upload/download large model parameters over cellular networks is a greater bottleneck than local computation.
  • The Availability Trap: Devices are typically only available for FL when idle, charging, and on WiFi. This creates a "participant bias" that researchers are still learning to model accurately.

Despite these gains, several challenges remain:

  • Theoretical Rigor: Many current selection methods are heuristic-based. There is a pressing need for theoretical guarantees that these methods won't lead to long-term model instability or bias.
  • Incentive Mechanisms: Future systems must consider why a client should participate, potentially incorporating incentive mechanisms to reward users for contributing their data and battery life.

Conclusion

Client selection is a critical lever for making Federated Learning practical at scale. By transitioning from random sampling to utility-aware mechanisms, we can build decentralized AI systems that are faster, more accurate, and more robust to the complexities of the real world.

Summary updated

Abstract

As a privacy-preserving paradigm for training Machine Learning (ML) models, Federated Learning (FL) has received tremendous attention from both industry and academia. In a typical FL scenario, clients exhibit significant heterogeneity in terms of data distribution and hardware configurations. Thus, randomly sampling clients in each training round may not fully exploit the local updates from heterogeneous clients, resulting in lower model accuracy, slower convergence rate, degraded fairness, etc. To tackle the FL client heterogeneity problem, various client selection algorithms have been developed, showing promising performance improvement. In this paper, we systematically present recent advances in the emerging field of FL client selection and its challenges and research opportunities. We hope to facilitate practitioners in choosing the most suitable client selection mechanisms for their applications, as well as inspire researchers and newcomers to better understand this exciting research topic.
BibTeX
@article{fu2023client,
  author = {Lei Fu and Huanle Zhang and Ge Gao and Huajie Wang and Mi Zhang and Xin Liu},
  title = {{Client Selection in Federated Learning: Principles, Challenges, and Opportunities}},
  journal = {IEEE Internet of Things Journal},
  year = {2023},
  url = {https://openreview.net/forum?id=YCtKL_9cEj},
}
Figure