Introduction
Standard Federated Learning (FL) enables collaborative model training without raw data sharing but is often hindered by massive communication overhead and high computational demands on edge devices. While Split Learning (SL) and Hierarchical FL have been proposed as solutions, they frequently suffer from "feature drift"—poorly structured intermediate data that leads to slow convergence. Furthermore, the requirement for cloud-to-client gradient backpropagation introduces significant latency and privacy risks.
To address these challenges, we present SHeRL-FL (Split and Hierarchical Representation Learning), a framework that integrates model splitting with edge-level contrastive representation learning to optimize both efficiency and accuracy.
The SHeRL-FL Architecture
SHeRL-FL rethinks the hierarchy of federated systems by assigning specific functional roles to different tiers.
The framework introduces three core innovations:
- Role-Aware Model Partitioning: The model is split based on semantics. Clients handle shallow layers for low-level features; Edge servers manage deeper layers for high-level representations; and the Cloud hosts task-specific modules like classifiers.
- Edge-Level Contrastive Learning: Edge servers utilize a margin-based contrastive loss to refine feature representations locally. By encouraging similarity between related samples and pushing dissimilar ones apart before they reach the cloud, the system mitigates feature drift.
- Gradient-Free Cloud-to-Client Communication: Contrastive loss is backpropagated directly from edge servers to clients. This decouples the lower-tier training from the cloud, eliminating the need for long-range gradient transmissions and slashing round-trip delays.
Performance and Efficiency
SHeRL-FL was evaluated on diverse datasets, including CIFAR-10/100 and medical imaging datasets like HAM10000 and ISIC-2018, under both IID and non-IID settings.
Key Results:
- Communication Efficiency: SHeRL-FL reduces data transmission by over 90% compared to centralized and standard Hierarchical FL, and by 50% compared to SplitFed.
- Superior Accuracy in Diverse Environments: In non-IID settings, SHeRL-FL consistently outperforms baselines. For example, on the HAM10000 dataset, it reached an F1 score of ~78%, significantly higher than traditional FL methods.
- Resource Optimization: Client RAM usage is maintained at a low ~380 MB, compared to ~580 MB for standard FL, making it highly suitable for resource-constrained IoT devices.
Conclusion and Future Directions
By treating the edge as a "feature refiner" rather than just a relay, SHeRL-FL achieves a breakthrough in hierarchical learning efficiency. Future work will explore adaptive partitioning to dynamically move split points based on network conditions and the integration of self-supervised learning to leverage unlabeled edge data.
Summary updated