Abstract
Federated Learning (FL) typically involves a group of clients collaborating to train a single global model. However, many real-world applications require the simultaneous training of multiple distinct models across a shared pool of heterogeneous clients. This paper introduces the Multiple-Model Federated Learning (MMFL) framework and addresses the challenges of task difficulty imbalance and client preference heterogeneity. The authors propose FedFairMMFL, a difficulty-aware allocation algorithm, and a novel Max-Min Fair Auction mechanism to ensure fair performance and participation across all tasks.
Introduction
In standard FL, the focus is on optimizing a single objective. In practice, a network of devices (e.g., smartphones or IoT sensors) might need to support various tasks simultaneously, such as heart rate monitoring alongside disease detection. This "Multiple-Model" setting introduces two primary hurdles:
- Heterogeneous Task Difficulty: Different models (e.g., complex image recognition vs. simple sensor data) require different amounts of data and computation to reach target accuracy.
- Client Preferences: Clients may be biased towards certain tasks due to personal interest or resource costs, leading to uneven training progress across the model suite.
Methodology: The FedFairMMFL Framework
To address these challenges, the authors propose a two-stage approach encompassing client recruitment and fair task allocation.
Difficulty-Aware Allocation (FedFairMMFL)
The core of the training process is the FedFairMMFL algorithm. Instead of uniform client distribution, it employs $\alpha$-fairness to dynamically allocate clients based on real-time task performance. By monitoring the current loss of each task, the algorithm assigns a higher probability of client allocation to the "harder" or worst-performing tasks. This ensures that training effort is concentrated where it is most needed, preventing difficult tasks from being neglected.
Max-Min Fair Auction
To manage client willingness and budget constraints, a truthful auction mechanism is integrated. This auction aims to maximize the "minimum take-up rate" across all tasks. By providing financial or resource-based incentives, the server ensures that even unpopular tasks receive sufficient client participation to progress.
Experimental Setup
The framework was evaluated using four computer vision benchmarks: MNIST, Fashion-MNIST, CIFAR-10, and EMNIST. These datasets represent a spectrum of difficulty levels. The experiments simulated non-i.i.d. data distributions, where each client holds only a subset of classes, reflecting the fragmented nature of real-world data.
Key Results and Insights
- Fairness and Accuracy: FedFairMMFL significantly improves the accuracy of the "worst-case" models without substantial degradation of average performance. By using loss as a proxy for difficulty, the system adapts to model requirements without prior knowledge of their architectures.
- Convergence and Variance: Difficult tasks reached target accuracy faster under FedFairMMFL compared to round-robin baselines. Furthermore, the variance in accuracy across different tasks was reduced, leading to more uniform outcomes.
- Auction Efficiency: The Max-Min Fair Auction proved more effective than standard budget-fair auctions in balancing client distribution, particularly when the server's budget was limited.
Conclusion and Future Work
This work highlights the importance of fairness in multi-task federated environments. While FedFairMMFL successfully balances task performance, future research could address client-side resource constraints—such as allowing powerful clients to train multiple models per round—and explore the trade-offs between task-level fairness and client-level fairness.
Summary updated