Adaptive Federated Learning for Heterogeneous Devices

50 concepts45 papers1 proposal

Standard federated learning (FL) operates on a clean mathematical abstraction: a central server distributes a uniform global model to a fleet of clients, each client computes gradients on its local data, and the server averages these updates synchronously.

This works beautifully in simulation, but breaks down immediately in the wild. Real-world edge networks are messy. As noted by Nicolas Kourtellis et al. in FLaaS: Federated Learning as a Service, older or lower-tier devices experience drastically longer execution times and higher power consumption when training. Forcing a uniform training workload across a diverse fleet introduces a severe bottleneck known as the straggler problem. The server must either wait for the slowest device to finish its local epochs—stalling the entire training round—or drop slow devices entirely, which discards valuable local data and introduces statistical bias.

The Dimensions of System Heterogeneity

To understand why uniform FL fails, we must look at the hardware constraints. As categorized by Stefanos Laskaridis et al. in Cross-device Federated Architecture Search, device heterogeneity spans three main physical dimensions:

  1. Computational Power: Variations in CPU/GPU clock speeds and available hardware accelerators.
  2. Memory Capacity: Strict RAM limits that cause Out-Of-Memory (OOM) crashes during backpropagation.
  3. Battery and Power: Constrained energy budgets that prevent mobile devices from running intensive optimization loops.

Furthermore, network conditions fluctuate wildly. A uniform strategy fails because it treats a resource-constrained microcontroller or an old smartphone the same as a high-end flagship device or a cloud node. To make federated learning viable at scale, the system must shift from a rigid, uniform paradigm to an adaptive, elastic one. Researchers have developed three primary mechanisms to achieve this.

Mechanism 1: Elastic Model Architectures

How do you train a model on a device that doesn't have the memory or compute to hold it? The intuition is simple: instead of forcing every client to train the exact same neural network, we can dynamically slice the global model into smaller, self-contained subnetworks tailored to each device's real-time capacity.

Global Model:  [====================] (Too large for older devices)

Subnetwork A:  [==========]           (Sent to low-tier client)
Subnetwork B:  [===============]      (Sent to mid-tier client)

In WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork Scheduling, Haozhe Su et al. propose a framework that dynamically resizes subnetworks based on a client's current CPU availability and wireless signal strength. Crucially, they use Fisher Information to determine which parameters are most critical to retain during the slicing process. This ensures that low-resource devices still contribute to the most impactful parts of the model without being crushed by the computational overhead.

Another angle is split learning under resource constraints. When fine-tuning large language models (LLMs) on edge devices, the memory footprint of backpropagation is often prohibitive. In SplitFrozen: Split Learning with Device-Side Model Frozen for Fine-Tuning LLM on Heterogeneous Resource-Constrained Devices, Jian Ma et al. show that freezing the device-side layers of an LLM and offloading parameter-efficient fine-tuning (like LoRA) to a server allows resource-constrained clients to participate in heavy workloads without exceeding their local hardware limits.

Mechanism 2: Tiered and Asynchronous Aggregation

If models cannot be resized, or if network bandwidth remains the primary constraint, the system can adapt the timing and grouping of updates rather than the model structure itself.

Traditional FL requires all clients to complete a fixed number of local iterations. To break this lockstep dependency, Shuyan Hu et al. introduce OFDMA-F²L: Federated Learning With Flexible Aggregation Over an OFDMA Air Interface, which allows clients to execute a variable number of local iterations based on their immediate computing capabilities and channel conditions before uploading.

Alternatively, clients can be grouped into distinct operational tiers. In FedDCT: A Dynamic Cross-Tier Federated Learning Framework in Wireless Networks, Youquan Xian et al. design a system that dynamically clusters devices into tiers based on their expected response times. By managing client selection and aggregation across these tiers independently, the server prevents fast clients from idling while waiting for slower tiers, drastically accelerating global convergence.

Mechanism 3: Co-optimizing Communication Precision and Payload

A third lever for handling heterogeneity is adjusting how data is transmitted and represented. A device with a weak network connection or low battery shouldn't send full-precision 32-bit floating-point weights.

The intuition here is to jointly optimize computation and communication. In Energy Efficient Federated Learning Over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmission, Liang Li et al. demonstrate that letting devices flexibly quantize their local model updates based on their individual battery levels and bandwidth constraints minimizes energy consumption without degrading global accuracy.

Similarly, Ngoc-Hieu Nguyen et al. exploit the low-rank structure of model updates in Towards Efficient Communication Federated Recommendation System via Low-rank Training, freezing a majority of the parameters and training only a small subset to shrink the payload size by over 90%, making it accessible for bandwidth-starved clients.

The Open Frontier

While these adaptive mechanisms mitigate the immediate friction of system heterogeneity, extreme disparities remain an open challenge. As highlighted in a comprehensive survey by Kilian Pfeiffer et al. (Federated Learning for Computationally Constrained Heterogeneous Devices: A Survey), when the resource gap between the most powerful and most constrained devices reaches 100x or 250x, understanding the exact mathematical contribution of low-resource devices to the global model's performance becomes highly non-trivial.

The ultimate goal of the field is moving away from rigid synchronization toward a completely fluid ecosystem where the global model dynamically morphs to fit the hardware topology it inhabits.

Key Foundational Work

Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation (2021) — contributed 1 concept elements to this direction.
Oort: Efficient Federated Learning via Guided Participant Selection (2021) — contributed 1 concept elements to this direction.
Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_device_heterogeneity_in_federated_learni_b8fecf49,
  title = {device heterogeneity in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/device-heterogeneity-in-federated-learning/rdr_b8fecf4982d74b65ad5c96aba63a0caa},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_device_heterogeneity_in_federated_learni_b8fecf49_v1,
  title = {device heterogeneity in federated learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/device-heterogeneity-in-federated-learning/rdr_b8fecf4982d74b65ad5c96aba63a0caa/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake