Resource Efficient Federated Learning on Heterogeneous Edge Devices

91 concepts85 papers2 proposals

Federated Learning (FL) promises a future where machine learning models are trained collaboratively without sensitive data ever leaving a user's device. However, a fundamental "tug-of-war" exists between the desire for high-performance, large-scale models and the harsh reality of edge hardware. As Ziwei Zhan et al. observe, smartphones, IoT sensors, and autonomous vehicles are often constrained by "measly compute power," limited battery life, and unreliable network bandwidth.

To bridge this gap, research has shifted from simple model averaging toward resource-aware frameworks that optimize the trade-offs between communication, computation, and energy consumption.

Balancing Communication and Computation

The primary bottleneck in FL is often the cost of "talking" (transmitting updates) versus "working" (local training). Liang Li et al. characterize this as a flexible trade-off where allowing heterogeneous devices to use different gradient compression levels and update frequencies can reduce energy consumption by up to 100x.

When models grow to millions of parameters, even simple updates can overwhelm residential or mobile networks. Minsu Kim et al. propose SpaFL, which utilizes sparse models to reduce the bandwidth footprint. Similarly, the authors of SparsyFed demonstrate that achieving extreme model sparsity (up to 95%) can slash communication costs by 19x while maintaining accuracy, effectively solving the "consensus problem" where diverse clients struggle to agree on a shared sparse structure.

Parameter-Efficient Fine-Tuning at the Edge

Training large foundation models, such as Vision Transformers (ViTs) or Large Language Models (LLMs), is typically infeasible for edge devices due to memory bottlenecks. Recent work focuses on Parameter-Efficient Fine-Tuning (PEFT) to minimize the local workload.

For instance, Azadeh Motamedi et al. introduced NormFit, which fine-tunes only the Pre-LayerNorm parameters of a Vision-Language Model. By updating just ~1,500 parameters, they reduce communication costs by 100x compared to standard adapter-based methods. In the realm of LLMs, Yebo Wu et al. proposed SMARTFED, which avoids training new modules entirely. Instead, it dynamically composes existing pre-trained LoRA experts, reducing energy consumption by 3.6x.

However, these efficiencies often come with a performance cost. In medical imaging, Naif Alkhunaizi et al. identified a "4% penalty," where every order of magnitude reduction in communication costs results in a roughly 4% drop in accuracy, highlighting that PEFT does not always lower the local memory requirements necessary for training.

Precision and Energy Optimization

Beyond reducing the number of parameters, researchers are optimizing the precision of the math itself. Zhiwei Li et al. demonstrated that 8-bit local training is sufficient for FL, provided the server aggregates updates using high-precision moving averages. This significantly lowers the computational and memory burden on the client.

Energy efficiency is also a matter of hardware-aware optimization. Minsu Kim et al. developed a framework for Quantized Neural Networks (QNNs) that balances quantization error against hardware energy savings. By optimizing bit-precision, they achieved energy reductions of 53% by accounting for the power-intensive nature of multiply-accumulate operations and data movement.

System-Level Orchestration

Finally, the field is moving toward sophisticated scheduling to handle device heterogeneity. Bing Luo et al. established a mathematical link between the number of participating clients and local iterations to minimize the total cost of learning time and energy.

For wireless environments, Jun Li et al. utilized Lyapunov optimization to jointly schedule clients and allocate resources, ensuring that training remains stable even when devices have limited battery and fluctuating connectivity. To support these advances, simulation tools like Protea, developed by Wanru Zhao et al., now allow researchers to profile client resources automatically, preventing out-of-memory errors and ensuring that FL protocols are grounded in the physical realities of edge hardware.

Key Foundational Work

Model Pruning Enables Efficient Federated Learning on Edge Devices (2019) — contributed 1 concept elements to this direction.
Training Keyword Spotting Models on Non-IID Data with Federated Learning (2020) — contributed 1 concept elements to this direction.
To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_resource_constraints_in_federated_learni_7f0b4bcb,
  title = {resource constraints in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/resource-constraints-in-federated-learning/rdr_7f0b4bcb0f4b4a6e9ab19534eb6fe28d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_resource_constraints_in_federated_learni_7f0b4bcb_v1,
  title = {resource constraints in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/resource-constraints-in-federated-learning/rdr_7f0b4bcb0f4b4a6e9ab19534eb6fe28d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake