Goal-Oriented Communications in Federated Learning via Feedback on Risk-Averse Participation

Shashi Raj Pandey, Van Phuc Bui, Petar PopovskiarXiv 2023 · arXiv

TL;DR

This paper introduces a goal-oriented communication framework for Federated Learning (FL) where risk-averse clients strategically decide when not to transmit their local updates. By leveraging semantic feedback from the Parameter Server (PS), this approach reduces unnecessary communication link utilization by up to 1.4× while maintaining competitive global model accuracy.

What Problem Does This Paper Address?

In standard Federated Learning (FL), a central Parameter Server (PS) coordinates a pool of distributed devices to train a machine learning model without sharing raw data. Typically, the PS selects a subset of devices to upload their local model updates in each round.

However, traditional client selection methods treat this purely as a server-side problem, ignoring the local incentives and data heterogeneity of individual devices. Poor local updates—caused by noisy, unrepresentative, or stale data—not only waste precious communication bandwidth and on-device compute but can also degrade the performance of the global model.

The authors reframe this as a goal-oriented (semantic) communication problem. Instead of blindly transmitting updates when called upon, devices should have the autonomy to decide if their contribution is actually valuable to the global goal, effectively choosing when not to transmit.

Figure 1: Risk-averse FL setting: (Left figure) Plain FL, and in (Right figure) device k is a risk-averse client, which drops itself from the FL process following feedback from PSat the base station.

Methods

The authors propose a closed-loop framework combining Multi-Arm Bandits (MAB), data valuation, and risk analysis:

  1. Risk-Averse Participation & Local Regret: Devices evaluate a "local regret function" to track how much the global model's performance drops on their local dataset over successive rounds. If the risk exceeds a certain threshold ($\delta$), the device strategically drops out of the training round to save resources.
  2. Server-Side Selection via CVaR-UCB: The PS uses an Upper Confidence Bound (UCB) bandit algorithm modified with Conditional Value-at-Risk (CVaR). This allows the server to account for the worst-case distribution of client contributions and calculate the average risk ($\theta$) across participating devices.
  3. Semantic Feedback: Along with the updated global model, the PS broadcasts semantic feedback (the calculated model regret and risk metrics) back to the selected devices.
  4. Shapley Value Data Valuation: To determine the actual worth of each client's update without looking at their data, the PS uses Shapley Value (SV) estimation to calculate marginal contributions.

Datasets

The framework was evaluated using the well-known MNIST dataset (70,000 28x28 grayscale images across 10 classes). The authors simulated a network of $K = 50$ devices under two distinct data distributions:

  • i.i.d. (independent and identically distributed): Where data is uniformly distributed across devices.
  • non-i.i.d.: Representing real-world statistical heterogeneity where devices hold highly skewed class distributions.

Key Insights and Ideas

  • Risk as Semantic Information: Instead of measuring communication efficiency purely in bits or packets, the paper uses "participation risk" as a semantic metric to quantify the relevance of data before transmission happens.
  • Empowering the Client: Shifting the transmission decision to the client side creates a more democratic FL ecosystem. Devices protect their own computational resources when they realize their data won't meaningfully improve the global model.
  • Sub-linear Regret: By incorporating CVaR into the UCB bandit formulation, the server successfully bounds the cumulative regret, ensuring stable and predictable convergence even when clients dynamically drop out.

Key Results

The proposed method (Alg. 1) was compared against standard FedSGD (with full and partial participation) and a naive UCB selection baseline.

  • Communication Efficiency: For a risk threshold of $\delta = 0.9$, the proposed algorithm achieved an average link consumption of 2.2 links per round, which is 1.4× lower than both the standard UCB and FedSGD baselines.
  • Convergence and Accuracy: Despite using significantly fewer communication links, the risk-averse framework reached the target 80% model accuracy in the exact same number of global communication rounds as the baselines in both i.i.d. and non-i.i.d. settings.
Figure 2: Comparative performance evaluation to reach 80% model accuracy for varying δ.

Limitations and Future Work

While the framework demonstrates clear communication gains, the authors note a few open challenges:

  • Distributed Coordination: Achieving fully distributed coordination of transmission strategies without introducing significant signaling overhead remains difficult.
  • Timeliness and Latency: The current model does not explicitly account for time-critical constraints or straggler effects in real-time networked systems.
  • Scalability: Future work will focus on scaling efficient data valuation techniques (like Shapley Value approximations) so they can be executed rapidly over massive client pools.

Summary updated

Abstract

We treat the problem of client selection in a Federated Learning (FL) setup, where the learning objective and the local incentives of the participants are used to formulate a goal-oriented communication problem. Specifically, we incorporate the risk-averse nature of participants and obtain a communication-efficient on-device performance, while relying on feedback from the Parameter Server (\texttt{PS}). A client has to decide its transmission plan on when not to participate in FL. This is based on its intrinsic incentive, which is the value of the trained global model upon participation by this client. Poor updates not only plunge the performance of the global model with added communication cost but also propagate the loss in performance on other participating devices. We cast the relevance of local updates as \emph{semantic information} for developing local transmission strategies, i.e., making a decision on when to ``not transmit". The devices use feedback about the state of the PS and evaluate their contributions in training the learning model in each aggregation period, which eventually lowers the number of occupied connections. Simulation results validate the efficacy of our proposed approach, with up to $1.4\times$ gain in communication links utilization as compared with the baselines.
BibTeX
@misc{pandey2023goal,
  author = {Shashi Raj Pandey and Van Phuc Bui and Petar Popovski},
  title = {{Goal-Oriented Communications in Federated Learning via Feedback on Risk-Averse Participation}},
  year = {2023},
  eprint = {2305.11633},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.11633},
}
Figure