Abstract
In decentralized machine learning, data imbalance across nodes often leads to a "tyranny of the majority," where global models favor large data contributors at the expense of smaller ones. This paper introduces the Adaptive Stackelberg Weighting Model (ASWM), a framework that treats federated learning as a Stackelberg Evolutionary Game. By modeling nodes as strategic players—a Leader (large node) and Followers (small nodes)—the system dynamically optimizes contribution weights to ensure fairness. Experimental results on MedMNIST datasets demonstrate that ASWM improves performance for underrepresented nodes by up to 2.7% in AUC with negligible impact on major contributors.
Introduction
Standard Federated Learning (FL) algorithms, such as FedAvg, typically weight node contributions based on local dataset size. While effective for independent and identically distributed (IID) data, this approach fails in real-world scenarios where data is non-IID (non-independent and identically distributed). For instance, in medical imaging, a large university hospital may dominate the training process, causing the resulting global model to perform poorly on unique data distributions from smaller rural clinics. This creates a "fairness gap," discouraging participation from smaller entities who gain little from the collective training.
The Stackelberg Evolutionary Game Framework
To address this, we reframe FL as a Stackelberg Game—a strategic model where a "Leader" (the node with the most data) moves first, and "Followers" (smaller nodes) respond. Since FL is an iterative process, this evolves into an Evolutionary Game where strategies adapt over time. Unlike traditional FL where nodes are passive, here they are active players with specific objectives.
We propose the Adaptive Stackelberg Weighting Model (ASWM). ASWM utilizes an actor-critic neural network and an experience replay buffer to predict optimal contribution weights ($C_k$). The Leader’s network anticipates Follower reactions to specific weightings, seeking a "Stackelberg Equilibrium" that balances collective performance with individual node needs.
Experimental Evaluation
The model was validated using three MedMNIST subsets: BloodMNIST (blood cell classification), DermaMNIST (skin lesion diagnosis), and BreastMNIST (breast ultrasound analysis). To simulate realistic imbalances, we applied a Dirichlet distribution to skew class proportions and introduced unique Gaussian noise patterns to represent hardware variability across nodes. This setup ensures that the model is tested against both label skew and feature distribution shift, common hurdles in decentralized medical AI.
Results and Discussion
ASWM consistently outperformed standard Federated Averaging (FedAvg) and Precision-Weighted FedAvg (PWFedAvg). Key findings include:
- Follower Gains: Underrepresented nodes experienced an average AUC increase of 2.713%, with some nodes seeing improvements as high as 5.97% compared to standard methods.
- Leader Stability: The performance cost to the Leader node was minimal, averaging only a 0.441% decrease. This suggests that the Leader actually benefits from the Followers' unique data (unfamiliar cases), formalizing a "mutual need" between nodes of different sizes.
- Dynamic Convergence: As shown in the weight evolution plots (Figures 1c and 1d), the system successfully reaches an equilibrium where weights stabilize. This dynamic adjustment round-by-round ensures a robust global model across all participating nodes regardless of their initial size.
Conclusion
By treating decentralized learning as a strategic game, ASWM effectively mitigates the "tyranny of the majority." While the sequential nature of Stackelberg games presents potential parallelism challenges, this work provides a foundational approach for achieving fairness in heterogeneous federated environments. Future research will focus on optimizing communication overhead and scaling the framework to larger networks with hundreds of nodes.
Summary updated