Federated learning allows multiple decentralized clients to collaboratively train a machine learning model without sharing their raw data. However, the standard optimization paradigm—exemplified by Federated Averaging (FedAvg)—faces a severe practical bottleneck: communication overhead. Transmitting millions or billions of model parameters back and forth between edge devices and a central server over wireless networks introduces massive latency, consumes significant energy, and frequently stalls due to straggling devices.
To make federated learning viable for real-world deployment, researchers have shifted focus from merely preserving privacy to drastically optimizing communication efficiency. This is achieved through two complementary strategies: shrinking the data payload exchanged per round and minimizing the total number of communication rounds required to reach convergence.
Shrinking the Payload per Round
The most direct way to alleviate network strain is to compress the information sent across the network. Traditional distributed training often relies on simple scalar quantization, but federated environments demand more aggressive, structurally aware compression techniques to handle bandwidth asymmetry and strict constraints.
Bi-Directional and Structural Compression
Many compression algorithms focus solely on the uplink (client-to-server), leaving the downlink unoptimized. To address this, Sixu Hu et al. developed a framework called Hybrid Gradient Compression (Practical Hybrid Gradient Compression for Federated Learning Systems), which exploits spatial and temporal redundancies across both the uplink and downlink. By compressing data in both directions simultaneously, they achieve compression ratios up to 15,000× on the uplink without degrading the model's convergence rate.
Similarly, Yongjeong Oh et al. proposed FedVQCS (Federated Learning via Vector Quantized Compressed Sensing), which combines dimensionality reduction with vector quantization. Rather than quantizing each gradient coordinate independently, vector quantization groups coordinates into vectors and maps them to the closest codebook vectors, preserving the underlying geometric structure of the updates even under stringent bandwidth limits.
Bypassing Parameter Averaging Entirely
When local data is highly heterogeneous (non-IID), averaging raw model parameters can cause "signal cancellation," where opposing local updates neutralize each other at the server. Some researchers avoid this by changing what is communicated.
For instance, Yuanhao Xiong et al. introduced FedDM (FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning), a method that replaces weight transmission with synthetic data transmission. Clients generate small, synthesized datasets via distribution matching that mimic their local data distributions. The server collects these compact synthetic sets and trains the global model directly, reducing per-round communication costs by roughly 80%.
In recommendation systems, Mingzhe Han et al. proposed FedCIA (FedCIA: Federated Collaborative Information Aggregation for Privacy-Preserving Recommendation), which aggregates item similarity matrices rather than raw model weights. This relational information is far more compact and stable, preventing signal cancellation and allowing clients with entirely different model architectures to participate in the same network.
Smoothing the Loss Landscape for Faster Convergence
While compression reduces the cost of a single round, the ultimate goal is to reduce the number of rounds. In non-IID environments, local models quickly drift away from the global optimum, converging to sharp, localized minima that generalize poorly when averaged.
Sharp Loss Landscape: Small client drift ──> Large gradient discrepancy ──> More rounds needed
Flat Loss Landscape: Small client drift ──> Small gradient discrepancy ──> Fewer rounds needed
By forcing local models to find flatter, more robust minima, updates from different clients remain structurally aligned, drastically accelerating global convergence.
Sharpness-Aware Minimization
Yan Sun et al. introduced FedSMOO (Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape), which integrates a globally synchronized Sharpness-Aware Minimization (SAM) objective with dynamic regularization. By actively seeking out flat regions in the loss landscape, FedSMOO ensures that local updates do not drastically diverge, reducing the required training rounds to less than a third of standard FedAvg.
To achieve a similar flattening effect with less computational overhead than SAM, Gaurav Kumar Nayak et al. proposed MAN (Minimizing Layerwise Activation Norm Improves Generalization in Federated Learning). This approach penalizes the norm of layerwise activations, bounding the Lipschitz constant of the network and naturally guiding the optimization toward flatter minima, which translates directly into fewer communication rounds.
Architectural and Normalization Refinements
The choice of model architecture also dictates convergence behavior in distributed settings. Standard Batch Normalization fails in federated learning because local batch statistics vary too wildly across clients.
Reza Nasirigerdeh et al. demonstrated that replacing batch normalization with Kernel Normalization (KernelNorm) stabilizes the training trajectory (Kernel Normalized Convolutional Networks for Privacy-Preserving Machine Learning). KernelNorm normalizes weights across the kernel dimension locally, eliminating dependency on batch statistics and allowing the global model to reach target performance levels in significantly fewer rounds.
Intelligent Client Selection and Network Topologies
Randomly sampling clients every round introduces statistical noise and exposes the system to the "straggler problem," where the entire round is bottlenecked by the slowest device. Modern frameworks structure how and when clients communicate based on their data and hardware profiles.
Clustering and Device Grouping
Instead of treating all edge nodes as a homogenous pool, frameworks can group devices strategically. Tian Liu et al. developed FLDG (Towards Fast and Accurate Federated Learning with non-IID Data for Cloud-Based IoT Applications), which uses Locality-Sensitive Hashing (LSH) to cluster IoT devices based on the similarity of their data feature maps. By selecting a balanced, representative set of devices from each cluster per round, they ensure that every gradient update reflects a global-like data distribution, cutting the total rounds needed to reach target accuracy by 80%.
To manage hardware heterogeneity, Youquan Xian et al. proposed FedDCT (FedDCT: A Dynamic Cross-Tier Federated Learning Framework in Wireless Networks), which dynamically tiers devices based on their real-time network response times. This cross-tier selection strategy prevents fast devices from idling while waiting for stragglers, yielding an average reduction in convergence time of 54.7%.
Decentralized and Asynchronous Topologies
Eliminating the central server entirely can remove a single point of communication failure. Wesley Houston Huff et al. introduced DHA-FL (DHA-FL: Enabling Efficient and Effective AIoT via Decentralized Hierarchical Asynchronous Federated Learning), an asynchronous, hierarchical edge framework. Because updates are processed asynchronously as soon as edge nodes complete their local steps, the system achieves up to an 8× convergence speedup over synchronous baselines when stragglers are present.
In highly fragmented or remote mesh networks where devices cannot reach an edge server, mobile infrastructure can bridge the gap. Matteo Zecchin proposed an optimization framework for using Unmanned Aerial Vehicles (UAVs) as flying relays (UAV-Aided Decentralized Learning over Mesh Networks). By calculating trajectories that prioritize connecting nodes that have not synchronized recently, the UAV effectively ferries model updates across isolated clusters, allowing decentralized networks to reach consensus rapidly without a fixed central orchestrator.
Key Foundational Work
Astraea: Self-Balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications (2019) — contributed 1 concept elements to this direction.
Learning Private Neural Language Modeling with Attentive Aggregation (2019) — contributed 1 concept elements to this direction.