QSFL: A Two-Level Uplink Communication Optimization Framework for Federated Learning

Liping Yi, Gang Wang, Xiaoguang LiuICML 2022 · OpenReview

Federated Learning (FL) enables multiple edge devices to collaboratively train a shared model without exposing their raw, private data. However, the "uplink" communication—transmitting model updates from edge devices to a central server—remains a significant bottleneck. Edge devices frequently operate on slow, expensive, and unreliable wireless connections.

While existing methods employ techniques like sparsification (sending only the most important gradients) or quantization (reducing the bit-width of updates), they often lack theoretical convergence guarantees or hit a "compression ceiling." Furthermore, most approaches ignore a crucial reality: not every client provides a useful update in every training round.

To address this, researchers have introduced QSFL (Qualification-based Sparse cyclic sliding segment Federated Learning), a two-level framework designed to drastically reduce communication overhead. By filtering out redundant clients and transmitting only small, sparsified segments of model updates, QSFL achieves up to an 890x compression ratio while maintaining model accuracy comparable to full-precision training.

A Two-Level Optimization Strategy

QSFL tackles communication inefficiency by optimizing at both the client level (determining who sends data) and the model level (determining what data is sent). The framework relies on two primary algorithms working in tandem:

  1. Qualification Judgment (QJ): This acts as the client-level filter. Instead of allowing all clients to upload their updates, the server evaluates them based on two metrics: Contribution (normalized local loss) and Relevance (how well the local gradient's sign aligns with the global model). Only the top $\alpha\%$ of "high-qualification" clients are permitted to upload. This saves massive amounts of bandwidth by ignoring redundant or low-quality updates.
  2. Sparse Cyclic Sliding Segment (SCSS): This is the model-level optimizer. Instead of sending a full model update, each qualified client sends only one segment (e.g., 1/6th of the model). To ensure the server eventually receives updates for all parameters, the segment ID "slides" cyclically each round. To push efficiency even further, a "sparse trick" (top-p% sparsification) is applied to each segment before transmission.
The complete workflow of the QSFL framework, detailing the Qualification Judgment (QJ) and Sparse Cyclic Sliding Segment (SCSS) stages.

The framework also includes a theoretical convergence proof and utilizes a Genetic Algorithm for hyperparameter searching to find the optimal balance between the number of segments, client sampling rate, and sparsification level.

Key Insights

  • The "Sliding" Reassembly: The core innovation of SCSS is the realization that the server doesn't need the full model from every client in every round. By having different clients send different segments, the server can reassemble a complete, updated global model from the collective pieces over time.
  • Redundancy is a Resource: The QJ algorithm recognizes that in large-scale FL, many clients have similar data or produce updates that don't significantly improve the model. Dropping these "redundant" clients doesn't just save bandwidth; it can actually stabilize training by focusing the global model on the most informative updates.
  • Theoretical Grounding: Unlike many heuristic-based compression schemes, the authors prove that QSFL converges over "wall-to-wall" time, demonstrating that extreme communication savings do not come at the cost of eventual model performance.

Experimental Results

The authors evaluated QSFL on two naturally non-iid (non-independent and identically distributed) datasets to simulate real-world edge environments: FEMINIST (a 62-class image classification dataset assigned to 36 clients) and Street (a real-world object detection task using YOLOv3 on images from 26 street cameras, assigned to 8 clients).

QSFL significantly outperformed standard baselines like FedAvg and specialized compression methods like signSGD and STC:

  • Massive Compression: On the FEMINIST dataset, QSFL reached a maximum compression ratio (CR) of 889.76x while maintaining a 78.08% mAP (mean Average Precision). For comparison, standard FedAvg only achieved a 19x CR.
  • Accuracy Preservation: When tuned for maximum accuracy, QSFL achieved 86.63% mAP on FEMINIST—slightly higher than FedAvg (86.54%)—while still maintaining a highly efficient 157.79x compression ratio.
  • Object Detection: On the more complex YOLOv3 Street task, QSFL achieved a 6.35x CR with 83.21% mAP, outperforming all other communication-efficient alternatives tested.
Comparison of compression ratios for different target mAP levels on the FEMINIST dataset, highlighting QSFL's massive efficiency gains.

Limitations and Future Work

While QSFL excels at uplink optimization, the authors note a few areas for future improvement. First, the framework currently focuses on the client-to-server path; the server still broadcasts the full model back to clients (the downlink bottleneck), which could be optimized in future iterations. Second, the SCSS segments are currently of equal size. The authors plan to explore dynamic segments, where the size of the uploaded segment adapts to the specific system resources (battery, bandwidth) of individual edge devices. Finally, while the paper addresses non-iid data, further work is needed to ensure robustness across highly heterogeneous hardware where computational speeds vary drastically between devices.

Summary updated

Abstract

In cross-device Federated Learning (FL), the communication cost of transmitting full-precision models between edge devices and a central server is a significant bottleneck, due to expensive, unreli...
BibTeX
@inproceedings{yi2022qsfl,
  author = {Liping Yi and Gang Wang and Xiaoguang Liu},
  title = {{QSFL: A Two-Level Uplink Communication Optimization Framework for Federated Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=Z-aPnv3E5h},
}
Figure