Inclusive Federated Learning for Heterogeneous Device Ecosystems

This direction is split into other directions: client exclusion bias. Its last version stays readable and citable here.

Version 1 of 1105 papers141 conceptsfrozen 2026-10-04

Federated Learning (FL) was conceived as a way to democratize AI training by moving computation to where the data lives: on billions of edge devices like smartphones and IoT sensors. However, a significant "participation gap" has emerged. In standard FL protocols, if a device is too slow, has too little memory, or suffers from a flaky network connection, it is often dropped from the training round to prevent it from becoming a "straggler" that stalls the entire global process.

While dropping stragglers makes training faster, researchers like Anit Kumar Sahu, Tian Li, and Manzil Zaheer have shown that systems heterogeneity and statistical heterogeneity are inextricably linked. When we exclude a device because of its hardware, we aren't just losing a processor; we are losing the unique data distribution that only that device holds.

The Hardware-Data Correlation

The core intuition behind this research direction is that hardware performance is often a proxy for user demographics. As Timothy Yang and colleagues at Google observed in large-scale Gboard deployments, strict requirements for RAM, battery, and Wi-Fi connectivity create a skewed training population.

If a model only learns from users with high-end flagship phones and stable fiber-optic internet, it becomes optimized for the behaviors and languages of those specific demographics. This creates a "fairness gap" where the resulting global model performs poorly for users in developing regions or those using older, budget hardware. Sebastian Caldas and H. Brendan McMahan argue that this systematic exclusion creates a demographic bias that undermines the very goal of representative AI.

Scaling the Memory Wall

Modern deep learning models are notoriously memory-hungry. A standard model might require several gigabytes of RAM just to store its gradients during training—a "memory wall" that many IoT devices cannot scale. Li Li notes that when resource-constrained devices are excluded due to these physical limits, the global model loses the diversity of the edge-tier data.

To solve this, researchers have moved toward "model-heterogeneous" training. Instead of forcing every device to train the same massive model, frameworks like FjORD, developed by Samuel Horváth and Mario Almeida, use Ordered Dropout to create nested sub-models. A low-end device might only train the "inner" 25% of the model, while a flagship phone trains the full 100%. This ensures that every device contributes to the shared knowledge base regardless of its RAM capacity.

Other approaches, such as FedLMT by Jiahao Liu and Yipeng Zhou, allow devices to train low-rank versions of the global model, providing a mathematically proven way to aggregate lightweight updates into a high-capacity global model.

Overcoming the Connectivity Barrier

Network bandwidth is the second major hurdle. In many geographic regions, upload speeds are too slow to transmit large model updates within a training window. Han Liu and Jia Liu highlight that these disparities in bandwidth effectively silence users in specific regions.

Solutions like ThrowRightAway (TRA) eliminate this bias by being "loss-tolerant". Instead of dropping a client when a packet is lost, the system incorporates partial updates, ensuring that even a weak signal contributes to the global consensus. Similarly, FedStaleWeight, proposed by Jeffrey Ma and Vijay Janapa Reddi, uses staleness reweighting to ensure that updates from slow devices aren't ignored, but are instead integrated fairly to prevent the model from drifting toward the data of "fast" users.

The Goal: No One Left Behind

The shift toward "Inclusive Federated Learning" represents a move from efficiency-first to representation-first design. As Ruixuan Liu and Lingjuan Lyu demonstrate with the InclusiveFL framework, incorporating data from resource-constrained devices doesn't just improve fairness—it improves the accuracy of the global model for everyone. By treating hardware diversity as a statistical asset rather than a technical nuisance, this research ensures that the future of decentralized AI is as diverse as the users it serves.

Papers in this version

Showing 60 of 105 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_inclusive_federated_learning_for_heterog_60ca7911,
  title = {Inclusive Federated Learning for Heterogeneous Device Ecosystems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/inclusive-federated-learning-for-heterogeneous-device-ecosystems/rdr_60ca791196aa4cda85c063d7ab8868b2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_inclusive_federated_learning_for_heterog_60ca7911_v1,
  title = {Inclusive Federated Learning for Heterogeneous Device Ecosystems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/inclusive-federated-learning-for-heterogeneous-device-ecosystems/rdr_60ca791196aa4cda85c063d7ab8868b2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}