Federated learning (FL) shifts model training from centralized data centers to distributed edge devices, allowing models to learn from private, localized data. However, synchronizing millions or billions of model parameters across thousands of clients over wireless networks creates a severe communication bottleneck. As surveyed by Seyedamin Pouriyeh et al. in Secure smart communication efficiency in federated learning, the central challenge of scaling FL lies in navigating the compression-accuracy tradeoff: reducing the number of transmitted bits without degrading the global model's convergence or predictive power.
Fortunately, neural network updates contain significant structural redundancy. Yo-Seb Jeon et al. demonstrate in Communication-Efficient Federated Learning over MIMO Multiple Access Channels that near-lossless classification accuracy can be maintained even when updates are compressed by a factor of five, provided the reconstruction error remains below a critical threshold. The core of modern FL research focuses on how to aggressively compress these updates while keeping reconstruction errors mathematically or functionally benign.
Mitigating Quantization Noise and Variance
The most straightforward way to compress updates is through quantization (reducing bit precision) and sparsification (masking out small gradients). However, naive compression introduces high variance and systematic bias, which accumulate over training rounds and derail convergence. As noted by Sahil Tyagi et al. in OmniFed: A Modular Framework for Configurable Federated Learning from Edge to HPC, standard algorithms like Top-k or QSGD offer varying balances of efficiency but often suffer from performance degradation when pushed to high compression ratios.
To combat this information loss, researchers design unbiased estimators and error-correction loops. For example, Louis Leconte et al. introduce a doubly stochastic Voronoi vector quantization method in $\texttt{DoStoVoQ}$: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning that eliminates quantization bias, allowing for drastic bandwidth reduction without sacrificing accuracy.
Similarly, Yongjeong Oh et al. combine dimensionality reduction with vector quantization in FedVQCS: Federated Learning via Vector Quantized Compressed Sensing to exploit the underlying sparsity of gradient vectors. In hierarchical network topologies, He Yang utilizes an explicit compression-correction mechanism in H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning to track and compensate for downstream aggregation errors at intermediate mediator nodes.
Alternative Representations: Factorization and Data Synthesis
Instead of compressing an existing dense weight matrix, a more radical approach alters how the model update is represented entirely.
One prominent strategy is structural factorization. In Trading-off Accuracy and Communication Cost in Federated Learning (AAMAS 2025), Mattia Jacopo Villani et al. introduce a framework called ZAMPLING. Instead of transmitting raw weights, ZAMPLING represents network weights as the product of a fixed, sparse random matrix and a highly compact, trainable probability vector. Because the optimization space is restricted to a low-dimensional probability simplex, client communication is reduced by up to 1024x with minimal impact on final accuracy.
Another paradigm shifts the communication medium from weights to functional data. Shengyuan Hu et al. propose FedSynth in FedSynth: Gradient Compression via Synthetic Data in Federated Learning, where clients transmit a small set of synthetic data points instead of gradient vectors. The server reconstructs the local update by running optimization steps on this synthetic data. This avoids the accuracy degradation typical of high-ratio weight masking by distilling the functional behavior of the local gradient into a highly compact data representation.
Information Theory and Physical-Layer Co-Design
The compression-accuracy tradeoff can be formalized using information theory. Huiyuan Yang et al. frame model aggregation as a lossy distributed source coding problem in Federated Learning With Lossy Distributed Source Coding: Analysis and Optimization. This framework maps out the fundamental theoretical bounds between the communication rate (in bits) and the distortion of the aggregated global model, enabling mathematically optimal rate allocation across clients.
In wireless edge networks, this trade-off directly intersects with physical channel properties. Merkebu Girmay et al. explore this in Over-the-air Aggregation-based Federated Learning for Technology Recognition in Multi-RAT Networks, utilizing Over-the-Air (OTA) computation. Instead of allocating separate orthogonal channels to each client, updates are transmitted simultaneously over the same frequency, allowing the physical superposition of radio signals to perform the mathematical aggregation automatically.
To combat the channel noise that threatens aggressive quantization in these setups, Hang Liu 0007 et al. incorporate Reconfigurable Intelligent Surfaces (RIS) in Reconfigurable Intelligent Surface Empowered Over-the-Air Federated Edge Learning to physically shape the wireless environment, optimizing the underlying communication-learning trade-off.
Exploiting Model Structure and Heterogeneity
Finally, communication efficiency can be achieved by transmitting only the specific subsets of a model that are relevant to the immediate optimization goal.
For instance, in clustered federated learning—where clients are grouped based on data similarity to handle non-IID data—Md Sirajul Islam et al. demonstrate in FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering that sending only the weights of the model's final layer is sufficient to accurately cluster clients. This drastically reduces the volume of transmitted data during the discovery phase while maintaining high personalization accuracy.
However, specialized architectures introduce their own communication challenges. As Koji Matsuda notes in Benchmark for Personalized Federated Learning, high-performing personalized FL methods—especially those relying on heavy knowledge distillation—frequently incur massive communication overheads compared to standard aggregation. Balancing localized model utility with network constraints remains an active area of optimization, a reality echoed in specialized domains like wireless traffic prediction frameworks developed by Chuanting Zhang et al. in Gradient Compression and Correlation Driven Federated Learning for Wireless Traffic Prediction.
Key Foundational Work
FedSynth: Gradient Compression via Synthetic Data in Federated Learning (2022) — contributed 2 concept elements to this direction.
H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning (2021) — contributed 1 concept elements to this direction.
$\texttt{DoStoVoQ}$: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning (2021) — contributed 1 concept elements to this direction.