Fair Resource Allocation in Federated Learning

Tian Li, Maziar Sanjabi, Virginia Smith

Federated Learning (FL) enables training statistical models across massive, heterogeneous networks of devices, such as mobile phones or IoT sensors, while keeping data localized. However, the standard objective of minimizing aggregate loss often leads to models that perform well on average but fail significantly for a minority of users. In their ICLR 2020 paper, Tian Li and colleagues introduce $q$-Fair Federated Learning ($q$-FFL), a framework designed to ensure a more uniform accuracy distribution across the network.

The Problem: The Tyranny of the Majority

Standard FL algorithms like FedAvg treat every data point or device as equally contributing to a global average. Because data in these networks is typically "Non-IID" (not identically distributed), a global model can become biased toward devices with larger datasets or more common data patterns. While average accuracy might remain high, individual devices may experience poor performance.

Conceptual goal of q-FFL: increasing the accuracy of the worst-performing devices to create a more uniform distribution.

Previous fairness approaches in machine learning often focused on protected groups, which is difficult to scale to millions of individual devices. Other methods, like Agnostic Federated Learning (AFL), were too rigid, focusing solely on the single worst-performing device and proving difficult to optimize at scale.

Proposed Method: $q$-FFL

The authors propose the $q$-FFL objective, inspired by "$\alpha$-fairness" in wireless resource allocation. The cost function is defined as:

$$f_q(w) = \sum_{k=1}^{m} \frac{p_k}{q+1} F_k(w)^{q+1}$$

The parameter $q$ allows for a tunable trade-off:

  • $q=0$: Recovers standard FedAvg.
  • Increasing $q$: Gives exponentially more weight to devices with higher local loss ($F_k$), forcing the model to improve performance for those lagging behind.
  • $q \to \infty$: Approaches a "minimax" solution, prioritizing the worst-performing device.

To solve this objective efficiently, the authors developed $q$-FedAvg. This solver includes a dynamic step-size adjustment that automatically adapts to the changing mathematical "smoothness" of the objective as $q$ varies, ensuring stable convergence without manual learning rate tuning for every $q$.

Experimental Evaluation

The framework was evaluated on several diverse federated benchmarks:

  • Synthetic: A controlled environment for tuning data heterogeneity.
  • Vehicle: Sensor data from 23 sensors for vehicle classification.
  • Sent140: Sentiment analysis of tweets from 1,101 accounts.
  • Shakespeare: Next-character prediction across 31 speaking roles.
  • Omniglot: Testing fairness in meta-learning ($q$-MAML).

Key Results: Achieving Uniformity

The experiments demonstrated that $q$-FFL effectively narrows the performance gap between devices. On average, the method reduced the variance of accuracies across devices by 45%.

Testing accuracy distributions across four datasets, comparing standard FedAvg (q=0) with the proposed q-FFL (q>0).

In the Vehicle dataset, increasing $q$ from 0 to 5 boosted the accuracy of the bottom 10% of devices from 43.0% to 69.9%, while the overall average accuracy slightly improved. Furthermore, $q$-FedAvg proved to be significantly faster than previous fairness baselines like AFL.

Conclusion and Future Directions

$q$-FFL provides a flexible and scalable solution for fair resource allocation in federated networks. By defining fairness as the uniformity of performance, it allows practitioners to balance average accuracy against the needs of the worst-performing devices. While choosing the optimal $q$ remains an empirical task, the framework's efficiency and adaptability make it a robust tool for real-world federated systems.

Summary updated

Abstract

Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.
BibTeX
@inproceedings{li2020fair,
  author = {Tian Li and Maziar Sanjabi and Virginia Smith},
  title = {{Fair Resource Allocation in Federated Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=Hl-ZfeSPLe5},
}
Figure