SHeRL-FL: When Representation Learning Meets Split Learning in Hierarchical Federated Learning

Dung T. Tran, Nguyen B. Ha, Van-Dinh Nguyen, Kok-Seng WongarXiv 2025 · DOI · arXiv

Introduction

Standard Federated Learning (FL) enables collaborative model training without raw data sharing but is often hindered by massive communication overhead and high computational demands on edge devices. While Split Learning (SL) and Hierarchical FL have been proposed as solutions, they frequently suffer from "feature drift"—poorly structured intermediate data that leads to slow convergence. Furthermore, the requirement for cloud-to-client gradient backpropagation introduces significant latency and privacy risks.

To address these challenges, we present SHeRL-FL (Split and Hierarchical Representation Learning), a framework that integrates model splitting with edge-level contrastive representation learning to optimize both efficiency and accuracy.

The SHeRL-FL Architecture

SHeRL-FL rethinks the hierarchy of federated systems by assigning specific functional roles to different tiers.

The SHeRL-FL architecture featuring role-aware model partitioning and edge-level contrastive loss.

The framework introduces three core innovations:

  1. Role-Aware Model Partitioning: The model is split based on semantics. Clients handle shallow layers for low-level features; Edge servers manage deeper layers for high-level representations; and the Cloud hosts task-specific modules like classifiers.
  2. Edge-Level Contrastive Learning: Edge servers utilize a margin-based contrastive loss to refine feature representations locally. By encouraging similarity between related samples and pushing dissimilar ones apart before they reach the cloud, the system mitigates feature drift.
  3. Gradient-Free Cloud-to-Client Communication: Contrastive loss is backpropagated directly from edge servers to clients. This decouples the lower-tier training from the cloud, eliminating the need for long-range gradient transmissions and slashing round-trip delays.

Performance and Efficiency

SHeRL-FL was evaluated on diverse datasets, including CIFAR-10/100 and medical imaging datasets like HAM10000 and ISIC-2018, under both IID and non-IID settings.

Comparison of F1 Score vs. Communication Rounds on CIFAR-100, demonstrating SHeRL-FL's superior convergence and accuracy over baselines.

Key Results:

  • Communication Efficiency: SHeRL-FL reduces data transmission by over 90% compared to centralized and standard Hierarchical FL, and by 50% compared to SplitFed.
  • Superior Accuracy in Diverse Environments: In non-IID settings, SHeRL-FL consistently outperforms baselines. For example, on the HAM10000 dataset, it reached an F1 score of ~78%, significantly higher than traditional FL methods.
  • Resource Optimization: Client RAM usage is maintained at a low ~380 MB, compared to ~580 MB for standard FL, making it highly suitable for resource-constrained IoT devices.

Conclusion and Future Directions

By treating the edge as a "feature refiner" rather than just a relay, SHeRL-FL achieves a breakthrough in hierarchical learning efficiency. Future work will explore adaptive partitioning to dynamically move split points based on network conditions and the integration of self-supervised learning to leverage unlabeled edge data.

Summary updated

BibTeX
@misc{tran2025sherl,
  author = {Dung T. Tran and Nguyen B. Ha and Van-Dinh Nguyen and Kok-Seng Wong},
  title = {{SHeRL-FL: When Representation Learning Meets Split Learning in Hierarchical Federated Learning}},
  year = {2025},
  eprint = {2508.08339},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.08339},
}
Figure