Decentralized Federated Learning With Energy Harvesting Devices

Kai Zhang, Xuanyu Cao, Khaled B. Letaief

TL;DR

This paper introduces a sustainable framework for Decentralized Federated Learning (DFL) by integrating energy harvesting (EH) at the edge. To manage the unpredictable nature of harvested energy without a central coordinator, the authors propose a localized policy iteration algorithm that achieves near-optimal scheduling and power control using only two-hop neighbor information.

What Problem Does This Paper Address?

Federated Learning (FL) typically relies on a central server to aggregate model updates. Decentralized Federated Learning (DFL) removes this bottleneck by allowing devices to exchange updates directly via device-to-device (D2D) links. While DFL improves privacy and reliability, it is incredibly energy-intensive. Edge devices (like sensors or phones) often have limited batteries that deplete quickly during local training and wireless transmission.

Energy Harvesting (EH)—drawing power from solar, wind, or thermal sources—offers a path to "perpetual" operation. However, EH is stochastic and unpredictable. In a decentralized system with no "boss" server, how do devices decide when to participate and how much power to use for transmission without causing massive interference or draining their batteries? Traditional optimization methods (like standard Markov Decision Processes) suffer from the "curse of dimensionality," where the complexity grows exponentially with the number of devices, making them useless for large-scale networks.

Methods

The authors tackle this through a three-step approach:

  1. Convergence Analysis: They derive a new convergence bound for wireless DFL that explicitly accounts for partial device participation and packet drops caused by energy constraints. This bound reveals a "persistent error" term that doesn't vanish over time unless scheduling and power are optimized.
  2. Multi-Agent MDP Formulation: They model the network as a multi-agent Markov Decision Process (MDP). Each device’s state includes its current battery level and the channel gains to its neighbors.
  3. Decentralized Policy Iteration: This is the core innovation. Instead of requiring a central controller to know every device's state, they propose an algorithm where devices only exchange Q-values and policies with their two-hop neighbors.
  4. They use a multiplicative-weights update rule to refine local policies.
  5. They prove that this localized approach is asymptotically optimal, meaning it eventually converges to the same performance as a perfect, all-knowing centralized controller.

Datasets

The study validates its approach using a combination of machine learning and environmental data:

  • Learning Tasks: Image classification using Fashion-MNIST (FMNIST) and CIFAR-10. They tested both IID (Independent and Identically Distributed) and non-IID data distributions to simulate real-world data heterogeneity.
  • Energy Data: A real-world solar irradiance dataset (recorded at Elizabeth City State University) to simulate realistic energy harvesting patterns.
  • Channel Models: Rayleigh fading modeled via finite-state Markov channels to simulate realistic wireless environments.

Key Insights and Ideas

  • The Two-Hop "Sweet Spot": The authors found that information from two hops away is the magic threshold. One-hop information is insufficient for effective coordination, while three hops increase communication overhead and computational complexity without providing significant accuracy gains.
  • Long-term vs. Myopic Planning: The results highlight that "myopic" strategies—which only try to maximize performance in the current time slot—eventually fail because they lead to frequent battery depletion. The MDP approach allows devices to "save" energy for future slots when channel conditions might be better.
  • Decentralization without Sacrifice: The most surprising finding is the theoretical and empirical proof that you don't need a central server to reach global optimality. Localized consensus on Q-values effectively "steers" the whole network toward a global optimum.

Key Results

  • Near-Optimal Performance: In simulations on FMNIST, the proposed decentralized algorithm closely matched the performance of the "Centralized Policy Iteration" (the theoretical upper bound).
  • Accuracy Gains: On FMNIST (IID), the proposed two-hop method achieved 84.2% accuracy, significantly outperforming "No communication" (75.9%) and "One-hop" (73.3%) variants.
  • Robustness to Non-IID Data: Even with highly heterogeneous data, the algorithm maintained a strong lead, reaching ~80% accuracy compared to ~62% for one-hop coordination.
  • Battery Efficiency: The algorithm successfully avoided the "death spirals" seen in uncoordinated policies, where devices blast power until they hit zero, leading to dropped model updates and stalled learning.

Limitations and Future Work

  • Tabular Complexity: The current algorithm uses a tabular approach for Q-values. While localized to two-hop neighborhoods, the state space could still be large in very dense networks. The authors suggest using function approximation (like Deep Q-Networks) as a future step to handle even larger state-action spaces.
  • Static Topology: The paper assumes a relatively stable network topology during the offline policy computation phase. Future work could explore how the system adapts to highly mobile nodes where the "two-hop neighborhood" changes frequently.
  • Offline Overhead: While the online execution is lightweight, the initial offline policy iteration requires several rounds of message passing ($R$ iterations). Finding ways to reduce this initial "setup" cost would be beneficial for dynamic environments.

Summary updated

Abstract

Decentralized federated learning (DFL) enables edge devices to collaboratively train models through local training and fully decentralized device-to-device (D2D) model exchanges. However, these energy-intensive operations often rapidly deplete limited device batteries, reducing their operational lifetime and degrading the learning performance. To address this limitation, we apply energy harvesting technique to DFL systems, allowing edge devices to extract ambient energy and operate sustainably. We first derive the convergence bound for wireless DFL with energy harvesting, showing that the convergence is influenced by partial device participation and transmission packet drops, both of which further depend on the available energy supply. To accelerate convergence, we formulate a joint device scheduling and power control problem and model it as a multi-agent Markov decision process (MDP). Traditional MDP algorithms (e.g., value or policy iteration) require a centralized coordinator with access to all device states and exhibit exponential complexity in the number of devices, making them impractical for large-scale decentralized networks. To overcome these challenges, we propose a fully decentralized policy iteration algorithm that leverages only local state information from two-hop neighboring devices, thereby substantially reducing both communication overhead and computational complexity. We further provide a theoretical analysis showing that the proposed decentralized algorithm achieves asymptotic optimality. Finally, comprehensive numerical experiments on real-world datasets are conducted to validate the theoretical results and corroborate the effectiveness of the proposed algorithm.
BibTeX
@misc{zhang2026decentralized,
  author = {Kai Zhang and Xuanyu Cao and Khaled B. Letaief},
  title = {{Decentralized Federated Learning With Energy Harvesting Devices}},
  year = {2026},
  eprint = {2602.14051},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.14051},
}
Figure